Math, when you need it

What you'll learn

Linear algebra · 10 lessons, about 5 hours

1. Vectors FreeA list of numbers as a point or an arrow in space.
2. Vector arithmetic and lengthAdd, scale and measure vectors.
3. Dot product and cosine similarityThe one operation behind similarity search and attention.
4. MatricesTables of numbers that hold datasets and model weights.
5. Matrix multiplicationMany dot products at once: what a neural network layer computes.
6. Matrices as transformationsA matrix stretches, rotates and projects space.
7. Transpose, identity and inverseThe matrix operations you'll see in every formula.
8. TensorsArrays with more axes: batches, sequences, heads.
9. Eigenvectors (intuition)The directions a transformation leaves pointing the same way.
10. SVD and PCACompress data to its most important directions.

Calculus · 8 lessons, about 4.5 hours

1. Functions and graphsRead a graph and see how an output depends on an input.
2. Exponentials and logarithmsGrowth, decay, and why logs turn products into sums.
3. DerivativesThe slope at a point: how fast the output changes.
4. Derivative rulesPower, sum and product rules, so you rarely compute from scratch.
5. The chain ruleDerivatives of functions inside functions: the heart of backprop.
6. Partial derivativesChange one input at a time when there are many.
7. GradientsThe vector of all partial derivatives: which way is uphill.
8. Optimization by walking downhillRepeatedly step against the gradient to find a minimum.

Probability & statistics · 8 lessons, about 4.5 hours

1. Probability basicsEvents, outcomes and the rules for combining them.
2. Conditional probability and BayesUpdate beliefs when new evidence arrives.
3. DistributionsRandom variables and the shapes their outcomes take.
4. Mean, variance and spreadExpected value, variance and standard deviation.
5. The normal distributionThe bell curve, and why it shows up everywhere.
6. Samples and confidenceEstimate from a sample and say how sure you are.
7. A/B tests and significanceTell a real lift from noise.
8. Maximum likelihoodChoose the parameters that make the data most probable.

Information · 3 lessons, about 1.5 hours

1. SoftmaxTurn any list of scores into probabilities.
2. Entropy and surpriseMeasure uncertainty in bits.
3. Cross-entropy and KL divergenceThe loss function language models are trained on.

Before you start

Lessons here build on an earlier phase:

• Phase 0: Tools

The placement test can mark off what you already know, so you only study what's new to you.

What you'll build

Cross-entropy and KL divergence

The loss function language models are trained on. Read the tutorial.

• Compute cross-entropy between a true distribution and a model's prediction
• Show that with one correct answer, cross-entropy is −log of the probability given to it
• Compute KL divergence and explain it as the extra cost of using the wrong distribution

When you finish

Finish every lesson in this phase, or show you know it in the placement test, and you earn a certificate of completion, "Math, when you need it (Phase 2)", with its own link you can share.