LearnAISign in
Phase 3 · 14 lessons · 44 h · about 5 weeks

📈 Classical Machine Learning

Learning from data, with the math that tells you when it works

Lessons

3.1What is learning? ERM, generalization and the ML workflowThe one idea every model in this path is a special case of2 h 30 min3.2Linear regressionLeast squares three ways: algebra, geometry and probability⚡ demo · 3 h3.3Logistic regression and softmax classificationTurning a linear score into a probability — and why that is a neuron⚡ demo · 3 h3.4k-Nearest NeighborsLearning by remembering — and the ancestor of vector search⚡ demo · 2 h 30 min3.5Evaluating models: metrics, validation and the bias–variance trade-offHow good is it, compared to what, and how sure are you?⚡ demo · 3 h 30 min3.6Regularization and generalizationThree views of one idea: constraint, shrinkage and prior⚡ demo · 3 h3.7Naive Bayes and generative vs discriminative modelsClassify by modelling how the data were made2 h 30 min3.8Decision treesLearning by asking the most informative question first2 h 30 min3.9Ensembles: bagging, random forests and gradient boostingMany weak opinions, combined carefully, beat one strong one3 h 30 min3.10Support vector machines and kernelsThe widest street, and the trick that makes it curve3 h3.11Clustering: k-means, GMMs/EM, DBSCAN and hierarchicalFinding structure when nobody gave you labels⚡ demo · 3 h3.12Dimensionality reduction: PCA, t-SNE and UMAPFinding the few directions that matter⚡ demo · 3 h3.13Feature engineering, data quality and data-centric MLThe unglamorous work that decides whether anything else matters3 h3.14Project: end-to-end tabular ML with scikit-learnFrom a CSV to a portfolio-quality notebook6 h

About this phase

Before neural networks, there is learning itself: a model is a function chosen from a hypothesis space to minimize a loss on data, and everything else — generalization, overfitting, regularization, validation — follows from that one idea. This phase builds the full classical toolkit on that foundation. You will derive linear and logistic regression (and discover they are one-layer neural networks), meet k-nearest neighbors (the ancestor of every vector-search system), learn to evaluate models honestly with metrics, cross-validation and the bias–variance decomposition, and understand regularization as geometry and as Bayesian prior. Then the workhorses: Naive Bayes, decision trees, random forests and gradient boosting (still the state of the art on tabular data), support vector machines and kernels, clustering with k-means and mixture models, and dimensionality reduction with PCA, t-SNE and UMAP. The phase closes with feature engineering, data-centric practice and a complete end-to-end project in scikit-learn. These ideas are not a detour on the way to deep learning: cross-entropy, embeddings, retrieval, evaluation and regularization all return, unchanged, in Phases 4–10.

By the end you can

  • Frame any learning problem as empirical risk minimization, choose a sensible loss and validation protocol, and recognize overfitting, underfitting and leakage
  • Derive and implement linear regression, logistic/softmax regression, kNN and Naive Bayes from scratch, and explain each as a special case of ideas used later in deep learning
  • Evaluate classifiers and regressors with the right metrics (precision/recall/F1, ROC and PR curves, log loss, RMSE), k-fold and nested cross-validation, and the bias–variance decomposition
  • Explain regularization three ways — constrained optimization, shrinkage, and Bayesian priors — and tune it
  • Train, tune and interpret decision trees, random forests, gradient-boosted trees and SVMs, and know when each wins
  • Use clustering and dimensionality reduction to explore data and embeddings, and deliver an end-to-end tabular ML project with scikit-learn