Statistics usually runs forward: from the data to an average, an error bar or a chance. Planning runs it backward. How many test questions give a ±2-point error bar? What input tips a classifier to 90%? What temperature turns these scores into those chances? What chance does a loss of 0.7 stand for? Each answer is a formula from the statistics module with a different letter made the subject. The moves are this subcourse's: a square undoes a root, a log undoes an exponential, and a power undoes a log.
What you'll be able to do
Turn the mean around for a missing value or for n, the standard error and a margin around for the sample size (rounded up), and a margin around for z
Run the regression line backward: x̄ from the intercept, and the x that gives a target prediction
Undo the sigmoid, softmax and temperature: the input for a target chance, the score gap for a ratio of chances, and T
Turn bits and losses into chances: p = 2^(−s), n = 2^H, p = e^(−L), and perplexity
State the conditions: n rounded up, p strictly between 0 and 1, a slope that isn't 0, and logs only of numbers above 0
Quick check
Have a guess before you start. An average of 25 people has a standard error of 2. About how many people bring it down to 1?
50
100
12.5
26
Show the answer
100
The standard error is σ/√n, so n sits under a square root. Halving the error means doubling √n, from 5 to 10. That takes n from 25 to 100: four times the people.
The math: turning statistics around
Purpose. Solve the statistics module's formulas for a different letter: n, a missing value, z or an input.
The mean. x̄ = Σx/n: add the values, then divide by how many there are. Multiply both sides by n and you get the total: Σx = n·x̄. That one move frees two letters.
A missing value is the total minus the values you know. Four shuttle waits average 8 minutes, and three of them are 6, 9 and 7. The total is 4 × 8 = 32, so the lost one is 32 − 22 = 10.
n is the total divided by the mean: n = Σx/x̄. The trail club walked 96 km this month at 12 km a hike, so 96 ÷ 12 = 8 hikes. The condition: x̄ can't be 0.
The standard error. SE = σ/√n is how much the mean of n values wobbles from sample to sample (lesson 98). Here σ is the spread of single values. n gets two layers: a square root, then σ divided by it. Peel them in reverse. Swap √n and SE, so σ ÷ SE = √n. Then square both sides: n = (σ/SE)².
Quick check
From an earlier lesson: single waits have σ = 12 minutes. What is the standard error of the mean of 36 waits?
6
72
2
0.33
Show the answer
2
SE = σ/√n = 12 ÷ √36 = 12 ÷ 6 = 2.
A margin. A 95% range for a mean is x̄ ± m, with the margin m = z·σ/√n and z = 1.96. That adds one more layer on n: times z. Watch n = 400 go through it with σ = 10, then come back. The top row builds the margin. The bottom row starts from it and runs each opposite, right to left.
Try it now
Try n = 100: the margin doubles to 1.96. A quarter of the data, twice the margin.
Symbols and formula.
SE=nσ⇒n=(SEσ)2,m=nzσ⇒n=(mzσ)2
In plain words: n hides under a root in the bottom of a fraction. Swap it with the error to bring √n up, then square. Then round up to a whole number.
Rounding up. n counts people or test questions, so it's whole. With σ = 12 and a margin of 3, the formula gives (1.96 × 12 ÷ 3)² = 7.84² ≈ 61.47. Then 61 people leave the margin a little wider than 3, so you need 62. A planned n always rounds up, never to the nearest.
Freeing z. z counts standard errors, like a z-score (lesson 97). Multiply both sides of m = zσ/√n by √n, then divide by σ: z = m√n/σ, for σ above 0. It tells you how sure a fixed sample lets you be. 100 waits with σ = 12 have a standard error of 12 ÷ 10 = 1.2. A margin of 1.2 is z = 1.2 × 10 ÷ 12 = 1: only about 68% sure, by the 68–95–99.7 rule (lesson 93).
Named mistakes. Forgetting to square: σ ÷ SE is √n, not n. Rounding 61.47 down to 61. And doubling n to halve the error: halving it takes four times the data.
Quick check
m = z·σ/√n. Make n the subject.
n = √(zσ/m)
n = (zσ/m)²
n = zσ/m
n = (m/(zσ))²
Show the answer
n = (zσ/m)²
Swap √n and m: √n = zσ/m. Then square both sides: n = (zσ/m)².
Quick check
A fixed sample of 100 waits has σ = 12 minutes. What z gives a margin of 2.4 minutes?
0.5
2
20
0.2
Show the answer
2
z = m√n/σ = 2.4 × 10 ÷ 12 = 2. Two standard errors either side: about 95% sure, by the 68–95–99.7 rule.
The line, run backward. The least-squares line ŷ = wx + b (lesson 101) passes through the middle point (x̄, ȳ). Put that point in: ȳ = w·x̄ + b, so b = ȳ − w·x̄. Some books call the slope m; here it's w.
Which x̄? Add w·x̄ to both sides, take b away, then divide by w: x̄ = (ȳ − b)/w.
Which x gives a target? The tent shop's line is ŷ = 2.53t − 20.75: rentals from the temperature t in degrees. For 50 tents, add 20.75, then divide by 2.53: t = (50 + 20.75)/2.53 ≈ 27.96 degrees.
Conditions. w can't be 0. A flat line gives the same ŷ for every x, so no x picks out a target. And the answer is only as good as the data near it. The line was fit on days from 14 to 28 degrees. A target of 80 tents needs about 39.8 degrees, far outside what the shop has seen.
In your own words
Why does a planned sample size round up, even when the formula gives 61.02?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: plan n, and turn bits into chances
Purpose. Write each rearrangement as a function that rounds the way a plan needs.
Like 0.1 + 0.2 in lesson 7, 1.96 and 0.98 are stored a hair off in binary. So 1.96 × 10 ÷ 0.98 comes out as 20.000000000000004, and its square as 400.00000000000017. That tiny error is called float dust. math.ceil sees a number above 400 and jumps to 401, one more than you need. Rounding to 6 places first clears the dust, which sits far past the sixth decimal place. A real fraction like 61.4656 survives the rounding and still goes up to 62. A margin of 0 would need endless data, so the function returns None.
Quick check
What does this print?
import math
print(2 ** -3, math.log2(32), round(2 ** 2.5, 4))
0.125 5.0 5.6569
-8 5.0 5.6569
0.125 16.0 5.6569
0.125 5.0 6.25
Show the answer
0.125 5.0 5.6569
2 to the power −3 is 1 ÷ 8 = 0.125. log₂ 32 asks "2 to what power is 32?", which is 5. And 2 to the power 2.5 is about 5.66.
Bits, turned around. The surprise of an event with chance p is s = −log₂ p bits (lesson 105). A 1-in-8 event carries 3 bits. To free p, multiply by −1, giving log₂ p = −s. Then raise 2 to both sides: p = 2^(−s).
The entropy of n equally likely choices is H = log₂ n. Raise 2 to both sides: n = 2^H. So 5 bits means 32 choices. A result that isn't whole is fine: 2^2.5 ≈ 5.66 means "as unsure as choosing among about 5.66 equal options". That's how perplexity is read.
Conditions. Surprise is never below 0, so p never goes above 1. In Python, 2 ** -s frees p and math.log2 runs the way forward.
Quick check
An event carries 4 bits of surprise. What is its chance?
1/16
16
1/8
1/4
Show the answer
1/16
p = 2^(−s) = 2⁻⁴ = 1/16. Check: −log₂(1/16) = 4.
Drill
Fix the plans
n_for_se rounds to the nearest whole number, z_for leaves σ on top, and chance_from_bits drops a sign. Fix all three, and keep the rounding safe from float dust.
In the lesson, your code is checked as you write it.
In your own words
Why round to 6 places before math.ceil, and why not just use round on its own?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: test sets, logits, temperature and losses
Purpose. Four everyday questions in model work are statistics formulas turned around.
How big a test set? Each test question is right or wrong, so σ = √(p(1 − p)) for an accuracy p. The margin is m = z√(p(1 − p))/√n. Turned around: n = z²·p(1 − p)/m². For an accuracy near 0.9 and a ±2-point margin at 95%, n = 1.96² × 0.09 ÷ 0.02² ≈ 864.4, so 865 questions. Not sure of p? p = 0.5 gives the largest p(1 − p), 0.25, so it's the safe plan: 2,401 questions.
Which input gives a target chance? A logistic model gives p = σ(wx + b) (lesson 102). Undo the sigmoid with the logit from lesson 35, then undo the line: x = (ln(p/(1 − p)) − b)/w. A trail-sign classifier scores a sign's pointedness x as z = 2x − 6. (Here z is a model's score, as in the logistic lesson, not a margin's 1.96.) It reaches p = 0.9 at x = (ln 9 + 6)/2 ≈ 4.10.
Conditions. p is strictly between 0 and 1, and w isn't 0. If w is negative, "at least 0.9" flips to "x at most" (lesson 36).
The logit gap. Softmax gives pᵢ = e^(zᵢ)/Σ e^(zⱼ) (lesson 104). In a ratio of two chances the sum on the bottom cancels: pᵢ/pⱼ = e^(zᵢ − zⱼ). Take ln of both sides: zᵢ − zⱼ = ln(pᵢ/pⱼ). For arrow to be 3 times as likely as tent, its score must lead by ln 3 ≈ 1.10, whatever the other scores are.
The temperature. At temperature T every score is divided by T first, so pᵢ/pⱼ = e^((zᵢ − zⱼ)/T). Take ln, then swap T and the log: T = (zᵢ − zⱼ)/ln(pᵢ/pⱼ). Scores 2 and 1 with a wanted ratio of 2 need T = 1/ln 2 ≈ 1.44. The ratio can't be 1, since ln 1 = 0. And it must favor the higher score, or T comes out negative.
A loss as a chance. Cross-entropy on the right answer is L = −ln p, so p = e^(−L). A training loss of 0.7 is an average over many answers, so e^(−0.7) ≈ 0.50 is a typical chance for the right answer: if every right answer got 0.50, the loss would be about 0.7. The perplexity is e^0.7 ≈ 2.01: as unsure as a choice between two.
Softmax gives arrow 0.6 and tent 0.2. How far apart are their scores?
e³ ≈ 20.1
ln 3 ≈ 1.10
3
0.4
Show the answer
ln 3 ≈ 1.10
The ratio is 0.6 ÷ 0.2 = 3, and the score gap is ln 3 ≈ 1.10. The other scores don't matter: their sum cancels.
Quick check
What does this print?
import math
deftemperature_for(gap, ratio):
return gap / math.log(ratio)
print(round(temperature_for(1, math.e ** 2), 4), round(temperature_for(2, math.e), 4))
1.0 2.0
2.0 0.5
0.5 2.0
0.5 0.5
Show the answer
0.5 2.0
ln(e²) = 2, so a gap of 1 needs T = 1 ÷ 2 = 0.5: cooling stretches the gap. ln e = 1, so a gap of 2 needs T = 2: heating shrinks it.
Drill
Run the classifier and the loss backward
Write input_for(p, w, b), the x with σ(wx + b) = p (return None unless 0 < p < 1 and w isn't 0), chance_from_loss(loss), the chance a cross-entropy loss in nats stands for, and perplexity(loss).
import math
definput_for(p, w, b):
return math.log(p / (1 - p)) / w - b
defchance_from_loss(loss):
return math.exp(loss)
defperplexity(loss):
return math.exp(-loss)
print(input_for(0.9, 2, -6), chance_from_loss(0.7), perplexity(0.7))
In the lesson, your code is checked as you write it.
In your own words
Why does planning a test set with p = 0.5 never give too few questions?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
Learn to make any letter of any formula the subject: layers undone in reverse order, fractions, a letter that appears twice, powers, roots, quadratics, logs, inequalities, trigonometry, vectors, matrices, calculus, probability, statistics and the formulas inside AI models, each checked in Python.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write