LearnAISign in
Phase 1 · 10 lessons · 31 h · about 4 weeks

∑ Mathematical Foundations

Linear algebra, calculus, probability, information and optimization — the pillars every model stands on

Lessons

1.1Vectors, dot products and normsThe geometry every model lives in2 h 30 min1.2Matrices as transformationsA matrix is what happens to a vector⚡ demo · 3 h1.3Eigen-things, SVD and low rankThe directions a matrix keeps, and the few that matter⚡ demo · 3 h 30 min1.4Derivatives, gradients and the chain ruleHow a tiny change in a parameter changes the loss3 h1.5Multivariable and matrix calculusJacobians, Hessians, the identities you need, and what autograd really computes3 h 30 min1.6Probability I: random variables and distributionsThe language of uncertainty, from coin flips to the Gaussian⚡ demo · 3 h1.7Probability II: Bayes, independence, MLE and MAPUpdating beliefs, and why minimizing a loss is maximizing a likelihood⚡ demo · 3 h 30 min1.8Statistics essentialsEstimators, error bars and the honest evaluation of models2 h 30 min1.9Information theory: entropy, cross-entropy and KL divergenceWhy cross-entropy is the loss of almost everything⚡ demo · 2 h 30 min1.10Optimization: from gradient descent to AdamHow a model actually learns, and why the step size is everything⚡ demo · 3 h 30 min

About this phase

Every model you will meet is written in four mathematical languages. Linear algebra describes data, weights and transformations: dot products, matrices, eigenvectors and the singular value decomposition. Calculus explains how a tiny change in a parameter changes the loss, which is all gradient descent needs: derivatives, gradients, the chain rule and matrix calculus. Probability and statistics give the vocabulary for uncertainty, likelihood, estimation and honest evaluation, and information theory connects prediction to compression and shows why cross-entropy is the loss of nearly everything. Optimization turns all of it into learning. This phase builds each pillar from the ground up with derivations, worked numeric examples and interactive demos, and ends every lesson by pointing at exactly where in the path the idea reappears.

By the end you can

  • Compute and interpret dot products, norms, projections, matrix products, eigenvectors and SVDs, and read a linear layer as a geometric transformation
  • Differentiate scalar, vector and matrix expressions with the chain rule, and derive the gradients behind linear regression, logistic regression and softmax cross-entropy
  • Work with distributions, expectation, variance, Bayes' theorem, maximum likelihood and MAP, and explain why minimizing a loss is maximizing a likelihood
  • Quantify uncertainty in estimates and evaluations with standard errors, confidence intervals and the bootstrap
  • Define entropy, cross-entropy, KL divergence and perplexity and connect them to the losses used in deep learning
  • Explain and implement gradient descent, SGD, momentum and Adam, and reason about step sizes, conditioning and schedules