Module 7's slips are the ones that reach reports. A mean pulled up by one slow request goes on a dashboard as "typical". A loss that hits a probability of exactly 0 prints inf, and a night of training is lost. A softmax without its exponential still adds up to 1, and quietly gives the wrong chances. A sample size planned on "twice the data, half the wobble" comes out four times too small. A model's training error is reported as what users should expect. Recall is reported where precision was meant, a two-point lead between models is announced before anyone asks whether it's luck, and a KL divergence is computed the wrong way around. Each runs without complaint. This review gathers the eight, each with the check that catches it, mixes problems from all of Module 7, and collects every rule on one sheet, ready for Module 8.
What you'll be able to do
Spot the common slips of Module 7: a mean pulled by an outlier, a log of zero, softmax without the exponential, twice the data expected to halve the wobble, training error reported as test error, precision and recall swapped, a lucky win taken as real, and KL turned around
Find and fix slips in statistics code that runs but gives a wrong answer
Mix problems from every lesson of the module, and run a short statistics program in your head, line by line
Use the module's formula sheet: each rule, what it says, and its Python line, and see where Module 7 sits in a model
Quick check
Have a guess before you start. A team reports "our model's error is 0.02" from the data it was trained on. What should you ask?
Nothing: 0.02 is low
Which library was used?
What's the error on held-back data?
How long did the training run take?
Show the answer
What's the error on held-back data?
Training error flatters a model; only held-back data shows what to expect on new cases (lesson 101).
Find the slip
Purpose. Eight slips cover most of Module 7's mistakes, and each has a quick check. The first five come from describing data, samples, regression, likelihood and softmax:
A mean pulled by an outlier. One 40-second request makes the "typical" time 10 seconds. Check: compare the mean with the median; a big gap means a lopsided column.
A log of zero. A probability that rounds to exactly 0 makes −log p infinite. Check: is the loss finite? Clip probabilities into [ε, 1 − ε] before the log.
Softmax without the exponential.z / z.sum() adds up to 1 but isn't softmax, and breaks on negative scores. Check: a score of 0 must still get a positive share.
Twice the data for half the wobble. The standard error divides by √n, so halving it takes four times the data. Check: σ/√(2n) is σ/√n divided by only 1.41.
Training error reported as test error. Check: which rows was the error computed on? Only held-back rows count.
Put numbers on the fourth slip before reading on.
Quick check
A standard error is 2 with 100 samples. How many samples bring it down to 1?
The last three slips come from the lessons on judging models and measuring a gap between distributions:
Precision and recall swapped. Both count the model's right yeses. Precision divides by every yes the model said, TP + FP; recall divides by every real yes, TP + FN. Check: a model that says yes to everything must get a recall of 1.
A lucky win taken as real. One model's lead over another on a test set can be noise. Check: compare them item by item, and see whether 0 falls outside the 95% range of the mean difference. And after k tries, the chance of at least one false win is 1 − 0.95ᵏ, not 0.05 × k.
KL turned around. KL(p ‖ q) weights each gap by the truth p and puts p on top of each ratio. Swapped, it's KL(q ‖ p), usually a different number. Check: which of the two is the truth?
Below are fifteen worked lines, with a comment saying what each should give. Eight have a slip.
Try it now
How the checks catch them. Each slip fails a one-line test. Printing the mean beside the median shows 10.4 against 3 for the response times. Printing the loss shows inf. Feeding the "softmax" a score of −1 gives a negative share. Computing σ/√(2n) shows the wobble shrank by 1.41, not 2. Asking "which rows?" of the reported error shows they were the training rows. A model that says yes to everything gets a "recall" below 1 from the swapped formula. Twenty tries at 0.05 each give a "chance" of 1. And naming the truth before writing a KL fixes its order. Predict the next one before you run it.
Quick check
What does this print?
import numpy as np
x = np.array([2.0, 3.0, 4.0, 3.0, 40.0])
print(x.mean(), np.median(x))
3.0 10.4
10.4 4.0
3.0 3.0
10.4 3.0
Show the answer
10.4 3.0
The 40 pulls the mean up to 10.4, above four of the five values; the median, the middle of 2, 3, 3, 4, 40, stays at 3.
Slips that hide in code. Like the earlier modules', these slips mostly run without an error. A softmax without its exponential passes the "adds up to 1" check for positive scores. A loss of inf only shows up when someone reads the printout, and nan can poison every weight within a step. The two drills below are slips of this kind: each runs, each gives a confident wrong answer, and one check exposes each.
Drill
A softmax with no exponential
This softmax adds up to 1 but gives the wrong chances, and a negative score gets a negative chance. Find the slip and fix it, keeping the safe shift.
import numpy as np
defsoftmax(z):
z = z - z.max()
return z / z.sum()
print(softmax(np.array([2.0, 1.0, 0.1])))
In the lesson, your code is checked as you write it.
In your own words
Which check would you run on any function that's meant to return a probability distribution, and why that one?
Write a sentence or two, then compare it with a good answer in the lesson.
Mixed practice
Purpose. Problems from across Module 7, mixed. Name each one's lesson before you start: describing data, samples, comparing models, likelihood, regression, logistic regression, scoring a classifier, softmax, entropy or KL divergence. If one stops you, the steps show the way, and the formula sheet in the last part has every rule in one place.
Play computer. The program below runs the module in small: a mean against a median, a standard error, precision and recall from the same counts, a lead read from paired differences (+1 where only B is right, −1 where only A is), and a KL divergence. Several lines sit where this review's slips hide. Work out every value in your head before you type it, the way Python prints it. .item() turns NumPy's one-number answers into plain Python numbers, so they print plainly.
Worked example
Worked example: evaluating a classifier with its checks built in
Score a two-class model on its training rows and its held-back rows, checking that every softmax row adds up to 1 and that no probability can hit log 0.
import numpy as np
defchances(S):
E = np.exp(S - S.max(axis=1, keepdims=True))
return E / E.sum(axis=1, keepdims=True)
defloss(S, y):
P = chances(S)
total = 0.0for i inrange(len(y)):
total = total - np.log(np.clip(P[i, y[i]], 1e-12, 1))
return P, total / len(y)
P, train = loss(np.array([[2.0, 0.5], [0.2, 1.5], [1.8, 0.1], [0.3, 2.2]]), np.array([0, 1, 0, 1]))
Q, test = loss(np.array([[1.0, 0.9], [0.4, 0.6], [2.5, 0.2]]), np.array([1, 1, 0]))
print("rows add to 1:", P.sum(axis=1).round(6).tolist())
print("train:", round(float(train), 3), "test:", round(float(test), 3))
It runs right here in your browser, with nothing to install.
The safe softmax keeps every row a distribution; the clip means a confident wrong answer costs a large but finite loss rather than inf. Then the honest number: the cross-entropy on the four training rows is 0.187, on the three held-back rows 0.479, more than twice as high. The test figure is the one to report, and the gap says the model fits its own rows better than new ones. Three checks, two lines each, catch most of this module's slips.
Quick check
What does this print?
import numpy as np
z = np.array([1.0, 0.0])
print((z / z.sum()).tolist(), (np.exp(z) / np.exp(z).sum()).round(3).tolist())
[1.0, 0.0] [1.0, 0.0]
[1.0, 0.0] [0.731, 0.269]
[0.5, 0.5] [0.731, 0.269]
[0.731, 0.269] [1.0, 0.0]
Show the answer
[1.0, 0.0] [0.731, 0.269]
Without the exponential, the score 0 gets no share at all; with it, e⁰ = 1 still earns 0.269 against e¹ = 2.718.
Quick check
A training loop prints loss: 0.41, 0.38, inf, nan, nan. Which slip is most likely?
The learning rate is too small
The model is already perfect
A probability hit 0, and its log became −inf
The data has an outlier in the mean
Show the answer
A probability hit 0, and its log became −inf
−log 0 is inf, and the next steps turn inf into nan. Clipping probabilities before the log, or computing the loss from the scores directly, prevents it.
Drill
A loss that blows up
This loss is meant to average −log of the chances the model gave the right answers, but one chance has rounded to 0.0 and the loss prints inf. Fix it by clipping the chances into [1e-12, 1] before the log.
import numpy as np
defnll(p_right):
returnfloat(-np.mean(np.log(p_right)))
print(nll(np.array([0.9, 0.0, 0.8])))
In the lesson, your code is checked as you write it.
In your own words
Which kind of problem in the mixed ladder did you have to stop and think about most? What told you which method to use?
Write a sentence or two, then compare it with a good answer in the lesson.
The formula sheet
Purpose. Every rule of Module 7 in one place, with what it says and its Python line.
lesson
rule
in plain words
in Python
97
x̄ = (1/n) Σ xᵢ; s = √((1/n) Σ (xᵢ − x̄)²); z = (x − x̄)/s
the model's extra surprise over the truth's own; 0 only when they match
p @ np.log2(p / q)
Each row is also a check: if an answer breaks a row, the slip is in that step.
Quick check
Which row of the rest of the sheet says why lowering a model's cross-entropy shrinks its KL divergence?
Lesson 106: H(p, q) = H(p) + KL(p ‖ q)
Lesson 103: F1 = 2PR/(P + R)
Lesson 102: the logistic slope p − y
Lesson 104: the safe softmax, shifted
Show the answer
Lesson 106: H(p, q) = H(p) + KL(p ‖ q)
The data fixes its own entropy H(p), so cross-entropy can fall only as far as the KL gap above it does: lowering one lowers the other by the same amount.
Where Module 7 lives in a model. Statistics is how a model is fit and how it's judged. Describing data scales its input numbers so no feature drowns the rest, and a batch's gradient wobbles like any average, by σ/√n. Likelihood picks its loss: squared error for a line, and for a classifier −log of the right answer's chance, the cross-entropy against the targets, which is the data's own entropy plus a KL gap. Logistic regression and softmax turn its scores into chances. And judging it is statistics too: held-back rows, precision and recall read from the confusion table, and an error bar on its lead over the last model. The map below lights all of it.
Coming up. First, lesson 108 turns this module's formulas around: sample sizes from error bars, inputs from chances, logit gaps from softmax, and chances from bits and losses. Then Module 8 has no new math: it runs everything at once. Lesson 109 trains one neuron by hand, from weighted sum to update. Lesson 110 stacks neurons into layers as matrix products, with softmax at the end. Lesson 111 trains that network with cross-entropy and the slope p − y flowing backward, lesson 113 builds attention from dot products and softmax, and lesson 118 is the capstone: a sign reader from the formulas alone.
In your own words
Pick the rule on the sheet you're most likely to slip on. What check would you use to catch the slip?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write