LearnAISign in
Phase 4 · 12 lessons · 35 h · about 5 weeks

🧠 Deep Learning Foundations

Stack dot products and nonlinearities, then learn every weight with the chain rule

Lessons

4.1From perceptron to MLP; universal approximationStacking dot products and nonlinearities until anything becomes learnable⚡ demo · 3 h4.2Activation functionsThe small nonlinear choice that decides whether gradients survive⚡ demo · 2 h4.3Loss functions catalogEvery loss is a negative log-likelihood in disguise — pick the right disguise⚡ demo · 2 h4.4Backpropagation, fully derivedThe chain rule on a graph, organized so every gradient costs one backward pass⚡ demo · 4 h4.5Optimizers and learning-rate schedulesFrom SGD to AdamW, and the schedule that matters more than the optimizer⚡ demo · 3 h4.6Initialization and normalizationKeeping every layer's signal at the right scale, forward and backward2 h 30 min4.7Regularization in deep netsDropout, weight decay, augmentation and why huge models still generalize⚡ demo · 2 h 30 min4.8Training dynamics and debuggingRead loss curves like an engineer and find the bug before it costs a GPU-week3 h4.9PyTorch II: real training pipelinesDatasets, modules, mixed precision, checkpoints and multi-GPU in one template3 h 30 min4.10Autoencoders, embeddings and representation learningEncoders compress, decoders reconstruct; meaning lives in between⚡ demo · 2 h 30 min4.11Reinforcement learning essentialsLearning from rewards: Bellman equations, Q-learning, policy gradients and PPO⚡ demo · 4 h4.12Compute, hardware and scale basicsBytes, FLOPs and bandwidth: the arithmetic behind every training decision⚡ demo · 2 h 30 min

About this phase

This phase turns the classical-ML toolbox into neural networks and gives you the complete mathematics of training them: the forward pass of a multilayer perceptron, the activation and loss functions that make it work, backpropagation derived line by line, the optimizers and schedules that actually converge, and the initialization, normalization and regularization tricks that separate a network that trains from one that doesn't. You then practise reading loss curves like a mechanic reads an engine, build a production-grade PyTorch training pipeline, meet encoders, decoders and embeddings for the first time, learn the reinforcement-learning essentials that return in RLHF and agents, and finish with the hardware and compute arithmetic behind every modern model. Everything in Phases 5–10 is a special case of what you learn here.

By the end you can

  • Write the forward pass of an L-layer network, count its parameters and explain why nonlinearity is essential
  • Derive backpropagation for a softmax classifier by hand and explain how autograd implements it
  • Choose and justify activation functions, losses, optimizers, schedules, initialization and normalization for a new problem
  • Diagnose a failing training run from its loss curves, gradient norms and activation statistics
  • Implement a reusable PyTorch training loop with mixed precision, checkpointing and evaluation
  • State the Bellman equations and the policy-gradient theorem, and connect them to RLHF and agents
  • Estimate memory, FLOPs, training time and cost for a model before you train it