Deep learning

What you'll learn

Neural networks · 7 lessons, about 4.5 hours

1. The artificial neuron FreeWeighted sum plus a nonlinearity.
2. Activation functionsReLU, sigmoid, GELU, and why nonlinearity matters.
3. Multi-layer networksStack layers to learn complex functions.
4. The forward passFollow numbers through a network to a prediction.
5. Computation graphsEvery calculation as a graph of small operations.
6. BackpropagationThe chain rule applied backwards to get every gradient.
7. Build an autograd engineWrite backprop yourself in ~100 lines (micrograd).

Training · 8 lessons, about 5 hours

1. Weight initializationStart weights at the right scale.
2. The training loopForward, loss, backward, step, repeat.
3. Mini-batches and SGDTrain on small random batches instead of everything at once.
4. OptimizersMomentum and Adam.
5. Normalization layersBatch and layer norm keep training stable.
6. DropoutRandomly switch off neurons to prevent overfitting.
7. Vanishing and exploding gradientsWhy deep networks were hard to train.
8. Debugging trainingRead loss curves and fix what's wrong.

Architectures · 4 lessons, about 3 hours

1. Residual connectionsSkip connections that let gradients flow.
2. Convolutional networksPattern detectors for images.
3. Recurrent networksProcess sequences step by step, and their limits.
4. Embedding layersLearned lookup tables that turn tokens into vectors.

Tooling · 4 lessons, about 3 hours

1. PyTorch tensorsNumPy-like arrays that run on GPUs.
2. PyTorch autograd and modulesLet PyTorch compute gradients; build models with nn.Module.
3. GPUs and computeWhy deep learning runs on graphics cards.
4. Capstone: train a classifierBuild, train and debug a network end to end.

Before you start

Lessons here build on earlier phases:

• Phase 1: Building with LLMs
• Phase 2: Math, when you need it
• Phase 3: Machine learning

The placement test can mark off what you already know, so you only study what's new to you.

What you'll build

Train a classifier

Build, train and debug a network end to end. Read the tutorial.

• Prepare tabular data for a neural network without leaking test information
• Run the sanity checks before trusting a training run
• Train a regularized network with early stopping on a validation metric

When you finish

Finish every lesson in this phase, or show you know it in the placement test, and you earn a certificate of completion, "Deep learning (Phase 4)", with its own link you can share.