Project: end-to-end tabular ML with scikit-learn

Pick a dataset and a question

Choose a tabular dataset with a real-world flavour and a few thousand to a few hundred thousand rows: the Kaggle House Prices regression, the Titanic classic, the Adult/Census Income classification data, a bike-sharing demand set, or a dataset from the Hugging Face Hub. Write one sentence that states the prediction target, the unit of prediction (one house, one passenger), the metric you will optimize (RMSE on log price; PR-AUC for a rare positive class — 3.5 Evaluating models: metrics, validation and the bias–variance trade-off) and how the model would be used. That sentence drives every later decision.

Step 1 — EDA with a purpose

Spend an hour, not a day. Check shapes, dtypes, missing-value rates per column, target distribution (skew → predict $\log(1+y)$), obvious duplicates, and class balance. Plot each important feature against the target. Hunt for leakage (3.13 Feature engineering, data quality and data-centric ML): any column that smells like a consequence of the target, any timestamp later than the prediction time. Write down three hypotheses about which features matter — you will check them against SHAP later.

Step 2 — Split and baseline first

Hold out a test set immediately (stratified for classification; time-based if there is a time dimension) and do not touch it until the end. Establish baselines before any modelling: predict the mean/majority class, then a regularized linear model (3.2 Linear regression, 3.3 Logistic regression and softmax classification, 3.6 Regularization and generalization). A baseline tells you whether your fancy model is actually learning something.

Step 3 — A leakage-safe pipeline

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.model_selection import cross_val_score, KFold

num_pipe = Pipeline([("impute", SimpleImputer(strategy="median")),
                     ("scale", StandardScaler())])
cat_pipe = Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                     ("onehot", OneHotEncoder(handle_unknown="ignore", min_frequency=20))])
pre = ColumnTransformer([("num", num_pipe, num_cols), ("cat", cat_pipe, cat_cols)])

model = Pipeline([("pre", pre),
                  ("gbdt", HistGradientBoostingRegressor(learning_rate=0.05,
                                                         max_iter=2000,
                                                         early_stopping=True))])
cv = KFold(n_splits=5, shuffle=True, random_state=0)
scores = cross_val_score(model, X_train, y_train, cv=cv,
                         scoring="neg_root_mean_squared_error")

Because imputation, scaling and encoding live inside the pipeline, every cross-validation fold fits them on its own training portion — no contamination. Swap the final estimator to compare models (ridge/logistic, random forest, gradient boosting — 3.9 Ensembles: bagging, random forests and gradient boosting) under exactly the same protocol, and keep a table of CV mean ± standard deviation (1.8 Statistics essentials).

Step 4 — Tune the GBDT

Use early stopping to choose the number of iterations, then search a small space with RandomizedSearchCV or Optuna: learning rate (0.02–0.2), max_leaf_nodes or depth, min_samples_leaf, L2 regularization, and feature/row subsampling (if using LightGBM/XGBoost). Expect a few percent improvement over defaults, not a miracle; if the gap between CV and training error is large, regularize harder; if both are high, engineer better features (3.13 Feature engineering, data quality and data-centric ML).

Step 5 — Interpret and analyze errors

Compute permutation importance and SHAP values on validation data; make the summary plot and two dependence plots for your top features. Then do error analysis: sort validation predictions by residual (or by loss) and read the 30 worst cases. Group them — a bad slice (one city, one rare category), a data error, a label error, an unavoidable ambiguity? Each group suggests a concrete action: a new feature, a data fix, or accepting the floor. Iterate once or twice; this loop is the heart of data-centric practice.

Step 6 — Final evaluation and write-up

Refit the chosen pipeline on all training data, evaluate once on the held-out test set, and report the number next to the CV estimate (they should agree within the CV standard deviation). Your notebook or repository should contain: problem statement and metric; data description and leakage checks; baselines table; final model with hyperparameters and seed; SHAP figures with two sentences of interpretation each; error-analysis findings; limitations and next steps; and a one-command way to reproduce everything (requirements.txt or uv lockfile, fixed seeds — 2.5 Tooling: Git, GPUs, experiments and the Hugging Face Hub).

Rubric for a portfolio-quality project

Stretch goals

Add a calibrated probability output and a decision threshold chosen for a stated cost trade-off; package the model behind a tiny Gradio or FastAPI app; try a tabular neural network and confirm (or refute) that the GBDT wins (3.9 Ensembles: bagging, random forests and gradient boosting); or enter the equivalent Kaggle competition and compare your score with the public leaderboard.

Where this shows up later

• The same project skeleton — framing, honest validation, error analysis, write-up — structures the vision project 5.10 Project: train and deploy an image classifier, the fine-tuning project 8.13 Project: fine-tune a small LLM with LoRA and ship a chat app, the RAG project 9.11 Project: a cited RAG assistant over your own documents, with evals and the agent project 10.12 Project: build a research agent with tools, an MCP server and an eval harness.
• The portfolio ladder and interview stories — 11.3 Portfolio project ladder, 11.4 ML and LLM system design and interviews.