Evaluating models: metrics, validation and the bias–variance trade-off

Evaluation is the hard part

A model is only as good as the number you trust about it, and that number answers three questions: how good (which metric), compared to what (which baseline and which data), and how sure (which error bars). The metric you can compute is rarely the objective you care about — click-through is not satisfaction, token-level loss is not helpfulness — so the first job is to pick a metric whose improvement would actually change a decision, and to say out loud what it misses (3.1 What is learning? ERM, generalization and the ML workflow).

The confusion matrix and what grows from it

For a binary classifier, every prediction lands in one of four cells: true positives (TP), false positives (FP, type I errors), false negatives (FN, type II errors), true negatives (TN). Everything else is a ratio of these.

Worked example. A fraud model on 1,000 transactions: $TP=40$, $FP=10$, $FN=20$, $TN=930$. Accuracy $=970/1000=0.97$. Precision $=40/50=0.80$. Recall $=40/60\approx0.667$. $F_1 = 2(0.8)(0.667)/(1.467)\approx 0.727$. Specificity $=930/940\approx0.989$, FPR $\approx 0.011$. Balanced accuracy $\approx 0.828$. Now the accuracy paradox: a model that flags nothing scores $940/1000 = 0.94$ accuracy and zero recall. Accuracy flattered the fraud model; recall and precision tell you it misses a third of the fraud and that one in five alerts is false. With more than two classes, compute per-class precision/recall and average them: macro (unweighted mean, every class counts equally), weighted (by class frequency), or micro (pool all cells, which for single-label problems equals accuracy).

Thresholds and curves

A probabilistic classifier (3.3 Logistic regression and softmax classification) plus a threshold is a hard classifier; move the threshold and every cell of the matrix changes. The ROC curve plots TPR against FPR as the threshold sweeps from 1 to 0; a random classifier is the diagonal, a perfect one hugs the top-left corner. Its area, AUC, has a clean meaning: the probability that a randomly chosen positive is scored higher than a randomly chosen negative — a measure of ranking quality independent of any threshold. The precision–recall curve plots precision against recall over the same sweep; its area (average precision) is the metric to prefer under heavy imbalance, because precision depends on the base rate while FPR does not: with 1% positives, a model with FPR $=0.01$ looks superb on ROC but produces as many false alarms as true detections. The interactive demo makes this concrete — drag the threshold between two overlapping score distributions and watch the matrix, precision, recall, $F_1$ and the ROC point move together.

Which threshold? The one that minimizes expected cost, $c_{FP}\cdot FP + c_{FN}\cdot FN$, where the costs come from the problem (a missed cancer versus an unnecessary biopsy), never from the default 0.5. If what you need is the probability itself — for ranking, for downstream decisions — evaluate it directly with log loss $-\frac1n\sum[y_i\log p_i+(1-y_i)\log(1-p_i)]$ or the Brier score $\frac1n\sum(p_i-y_i)^2$, and check calibration with a reliability diagram (3.3 Logistic regression and softmax classification).

Regression metrics

Report RMSE and MAE in the target's units with a baseline (predicting the mean gives $R^2=0$, RMSE $=\sigma_y$); a 2.1 RMSE means nothing without knowing that the mean-predictor gets 2.3.

Validation protocols and the leaks they prevent

A single holdout split (e.g., 80/20) is fine with abundant data but gives one noisy number. k-fold cross-validation splits the training data into $k$ folds (5 or 10), trains $k$ times holding out each fold, and reports the mean and standard deviation across folds — an estimate with a spread. Stratified folds preserve class proportions, essential for imbalance. Grouped folds keep all rows of one entity (patient, user, document) in the same fold, so the model cannot memorize the entity. Time-series splits train on the past and validate on the future (forward chaining), because shuffling time leaks the future into training. Nested cross-validation runs hyperparameter search inside each outer fold, so the outer score is not inflated by the search; use it when you report a number that others will rely on. Through all of this the test set remains untouched until the end (3.1 What is learning? ERM, generalization and the ML workflow).

Leakage is any route by which information about the validation or test targets reaches the training process: scaling, imputing or selecting features on the full dataset before splitting (2.3 pandas, data wrangling and plotting); near-duplicate rows across splits (common in text and images); features recorded after the outcome (an "account closed" flag predicting churn); and temporal leakage. Pipelines that fit every preprocessing step inside each fold (sklearn.pipeline.Pipeline inside cross_val_score) are the mechanical defence (3.14 Project: end-to-end tabular ML with scikit-learn).

The bias–variance decomposition

Why do models fail? Suppose $y = f(\mathbf{x}) + \varepsilon$ with $\mathbb{E}[\varepsilon]=0$, $\operatorname{Var}(\varepsilon)=\sigma^2$, and let $\hat f_{\mathcal{D}}$ be the model trained on a random training set $\mathcal{D}$. Fix a test point $\mathbf{x}$ and write $\bar f(\mathbf{x}) = \mathbb{E}_{\mathcal{D}}[\hat f_{\mathcal{D}}(\mathbf{x})]$ for the average prediction over training sets. The expected squared error at $\mathbf{x}$ is

The middle term vanishes because $\varepsilon$ is independent of $\mathcal{D}$ and has mean zero; the last is $\sigma^2$. Split the first by adding and subtracting $\bar f$:

Therefore

Bias is the systematic error of the average model — a line fit to a curve is biased everywhere. Variance is how much the fitted model wobbles from one training set to the next — a degree-15 polynomial or a 1-NN classifier changes completely when a few points change. Capacity moves you along the curve: more flexible models lower bias and raise variance, and the test error is U-shaped in between (3.1 What is learning? ERM, generalization and the ML workflow). $k$ in kNN (3.4 k-Nearest Neighbors), polynomial degree (3.2 Linear regression), tree depth (3.8 Decision trees) and the regularization strength $\lambda$ (3.6 Regularization and generalization) are all bias–variance dials. Averaging many high-variance models cancels variance — that is why random forests work (3.9 Ensembles: bagging, random forests and gradient boosting).

Math check. For kNN regression with noise $\sigma^2$, the variance term is exactly $\sigma^2/k$: averaging $k$ noisy neighbours divides the noise variance by $k$, while the bias grows as the neighbourhood widens. The trade-off is literal.

Learning curves

Plot training and validation error against the training-set size $n$ (or against epochs for iterative models).

Are the differences real?

A test accuracy is a sample proportion: with $n$ test examples and true accuracy $p$ its standard error is $\sqrt{p(1-p)/n}$ (1.8 Statistics essentials). At $p=0.9$ and $n=1{,}000$ that is $0.0095$ — a 95% interval of roughly $\pm 1.9$ percentage points — so two models at 90.2% and 90.7% are indistinguishable. Because both models are scored on the same examples, compare them paired: bootstrap the per-example differences (resample test items, recompute the difference 1,000 times, read off the interval), or use McNemar's test on the disagreement counts. For cross-validation, report mean $\pm$ standard deviation over folds and, when the decision matters, repeat with several shuffles. A difference smaller than the spread is not a result; this rule survives unchanged into LLM benchmarks, where leaderboards often rank models by gaps smaller than their standard errors (8.7 Evaluating LLMs).

Where this shows up later

• Regularization is the lever that trades bias for variance — 3.6 Regularization and generalization; bagging averages away variance — 3.9 Ensembles: bagging, random forests and gradient boosting.
• Pipelines inside cross-validation are the project discipline of 3.14 Project: end-to-end tabular ML with scikit-learn; grouped and temporal splits reappear in 3.13 Feature engineering, data quality and data-centric ML.
• Precision/recall become recall@k, MRR and nDCG for retrieval — 9.5 Retrieval quality: hybrid search, reranking, query transforms and metrics; faithfulness and answer relevance for RAG — 9.8 Evaluating and observing LLM apps.
• Standard errors, paired comparisons and contamination are the core of LLM evaluation — 8.7 Evaluating LLMs — and agent evaluation (pass@k vs pass^k) — 10.9 Evaluating agents.