We start this tutorial to reveal how you can harness TPOT to automate and optimize machine studying pipelines virtually. By working immediately in Google Colab, we make sure the setup is light-weight, reproducible, and accessible. We stroll by means of loading knowledge, defining a customized scorer, tailoring the search area with superior fashions like XGBoost, and organising a cross-validation technique. As we proceed, we discover how evolutionary algorithms in TPOT seek for high-performing pipelines, offering us transparency by means of Pareto fronts and checkpoints. Take a look at the FULL CODES here.
!pip -q set up tpot==0.12.2 xgboost==2.0.3 scikit-learn==1.4.2 graphviz==0.20.3
import os, json, math, time, random, numpy as np, pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import make_scorer, f1_score, classification_report, confusion_matrix
from sklearn.pipeline import Pipeline
from tpot import TPOTClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
SEED = 7
random.seed(SEED); np.random.seed(SEED); os.environ["PYTHONHASHSEED"]=str(SEED)We start by putting in the libraries and importing all of the important modules that assist knowledge dealing with, mannequin constructing, and pipeline optimization. We set a hard and fast random seed to make sure our outcomes stay reproducible each time we run the pocket book. Take a look at the FULL CODES here.
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=SEED)
scaler = StandardScaler().match(X_tr)
X_tr_s, X_te_s = scaler.remodel(X_tr), scaler.remodel(X_te)
def f1_cost_sensitive(y_true, y_pred):
return f1_score(y_true, y_pred, common="binary", pos_label=1)
cost_f1 = make_scorer(f1_cost_sensitive, greater_is_better=True)Right here, we load the breast most cancers dataset and cut up it into coaching and testing units whereas preserving class stability. We standardize the options for stability after which outline a customized F1-based scorer, permitting us to guage pipelines with a give attention to successfully capturing constructive circumstances. Take a look at the FULL CODES here.
tpot_config = {
'sklearn.linear_model.LogisticRegression': {
'C': [0.01, 0.1, 1.0, 10.0],
'penalty': ['l2'], 'solver': ['lbfgs'], 'max_iter': [200]
},
'sklearn.naive_bayes.GaussianNB': {},
'sklearn.tree.DecisionTreeClassifier': {
'criterion': ['gini','entropy'], 'max_depth': [3,5,8,None],
'min_samples_split':[2,5,10], 'min_samples_leaf':[1,2,4]
},
'sklearn.ensemble.RandomForestClassifier': {
'n_estimators':[100,300], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.ExtraTreesClassifier': {
'n_estimators':[200], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.GradientBoostingClassifier': {
'n_estimators':[100,200], 'learning_rate':[0.03,0.1],
'max_depth':[2,3], 'subsample':[0.8,1.0]
},
'xgboost.XGBClassifier': {
'n_estimators':[200,400], 'max_depth':[3,5], 'learning_rate':[0.05,0.1],
'subsample':[0.8,1.0], 'colsample_bytree':[0.8,1.0],
'reg_lambda':[1.0,2.0], 'min_child_weight':[1,3],
'n_jobs':[0], 'tree_method':['hist'], 'eval_metric':['logloss'],
'gamma':[0,1]
}
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)We outline a customized TPOT configuration that mixes linear fashions, tree-based learners, ensembles, and XGBoost, using fastidiously chosen hyperparameters. We additionally established a stratified 5-fold cross-validation technique, guaranteeing that each candidate pipeline is examined pretty throughout balanced splits of the dataset. Take a look at the FULL CODES here.
t0 = time.time()
tpot = TPOTClassifier(
generations=5,
population_size=40,
offspring_size=40,
scoring=cost_f1,
cv=cv,
subsample=0.8,
n_jobs=-1,
config_dict=tpot_config,
verbosity=2,
random_state=SEED,
max_time_mins=10,
early_stop=3,
periodic_checkpoint_folder="tpot_ckpt",
warm_start=False
)
tpot.match(X_tr_s, y_tr)
print(f"n⏱️ First search took {time.time()-t0:.1f}s")
def pareto_table(tpot_obj, ok=5):
rows=[]
for ind, meta in tpot_obj.pareto_front_fitted_pipelines_.gadgets():
rows.append({
"pipeline": ind, "cv_score": meta['internal_cv_score'],
"dimension": len(str(meta['pipeline'])),
})
df = pd.DataFrame(rows).sort_values("cv_score", ascending=False).head(ok)
return df.reset_index(drop=True)
pareto_df = pareto_table(tpot, ok=5)
print("nTop Pareto pipelines (cv):n", pareto_df)
def eval_pipeline(pipeline, X_te, y_te, title):
y_hat = pipeline.predict(X_te)
f1 = f1_score(y_te, y_hat)
print(f"n[{name}] F1(check) = {f1:.4f}")
print(classification_report(y_te, y_hat, digits=3))
print("nEvaluating high pipelines on check:")
for i, (ind, meta) in enumerate(sorted(
tpot.pareto_front_fitted_pipelines_.gadgets(),
key=lambda kv: kv[1]['internal_cv_score'], reverse=True)[:3], 1):
eval_pipeline(meta['pipeline'], X_te_s, y_te, title=f"Pareto#{i}")We launch an evolutionary search with TPOT, cap the runtime for practicality, and checkpoint progress, permitting us to reproducibly hunt for robust pipelines. We then examine the Pareto entrance to determine the highest trade-offs, convert it right into a compact desk, and choose leaders primarily based on the cross-validation rating. Lastly, we consider the perfect candidates on the held-out check set to substantiate real-world efficiency with F1 and a full classification report. Take a look at the FULL CODES here.
print("n🔁 Heat-start for additional refinement...")
t1 = time.time()
tpot2 = TPOTClassifier(
generations=3, population_size=40, offspring_size=40,
scoring=cost_f1, cv=cv, subsample=0.8, n_jobs=-1,
config_dict=tpot_config, verbosity=2, random_state=SEED,
warm_start=True, periodic_checkpoint_folder="tpot_ckpt"
)
strive:
tpot2._population = tpot._population
tpot2._pareto_front = tpot._pareto_front
besides Exception:
cross
tpot2.match(X_tr_s, y_tr)
print(f"⏱️ Heat-start additional search took {time.time()-t1:.1f}s")
best_model = tpot2.fitted_pipeline_ if hasattr(tpot2, "fitted_pipeline_") else tpot.fitted_pipeline_
eval_pipeline(best_model, X_te_s, y_te, title="BestAfterWarmStart")
export_path = "tpot_best_pipeline.py"
(tpot2 if hasattr(tpot2, "fitted_pipeline_") else tpot).export(export_path)
print(f"n📦 Exported finest pipeline to: {export_path}")
from importlib import util as _util
spec = _util.spec_from_file_location("tpot_best", export_path)
tbest = _util.module_from_spec(spec); spec.loader.exec_module(tbest)
reloaded_clf = tbest.exported_pipeline_
pipe = Pipeline([("scaler", scaler), ("model", reloaded_clf)])
pipe.match(X_tr, y_tr)
eval_pipeline(pipe, X_te, y_te, title="ReloadedExportedPipeline")
report = {
"dataset": "sklearn breast_cancer",
"train_size": int(X_tr.form[0]), "test_size": int(X_te.form[0]),
"cv": "StratifiedKFold(5)",
"scorer": "customized F1 (binary)",
"search": {"gen_1": 5, "gen_2_warm": 3, "pop": 40, "subsample": 0.8},
"exported_pipeline_first_120_chars": str(reloaded_clf)[:120]+"...",
}
print("n🧾 Mannequin Card:n", json.dumps(report, indent=2))We proceed the search with a heat begin, reusing the realized heat begin to refine candidates and choose the perfect performer on our check set. We export the successful pipeline, reload it alongside our scaler to imitate deployment, and confirm its outcomes. Lastly, we generate a compact mannequin card to doc the dataset, search settings, and the abstract of the exported pipeline for reproducibility.
In conclusion, we see how TPOT permits us to maneuver past trial-and-error mannequin choice and as a substitute depend on automated, reproducible, and explainable optimization. We export the perfect pipeline, validate it on unseen knowledge, and even reload it for deployment-style use, confirming that the workflow is not only experimental however production-ready. By combining reproducibility, flexibility, and interpretability, we finish with a sturdy framework that we are able to confidently apply to extra advanced datasets and real-world issues.
Take a look at the FULL CODES here. Be at liberty to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be happy to observe us on Twitter and don’t overlook to affix our 100k+ ML SubReddit and Subscribe to our Newsletter.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its recognition amongst audiences.