Probability slips are quieter than most. A list of chances that adds up to 0.9 still samples without an error. Two dependent events multiplied as if they were independent give a neat, wrong number. A test that catches 90% of the sick gets read as "a positive means 90% sick", and a whole decision rests on the swap. A simulation without a seed gives a different answer every run, so nobody can check it. This review gathers the module's five commonest slips, each paired with the one-line check that catches it, mixes problems from all of Module 6, and collects every rule on one sheet, ready for Module 7, where these chances meet real data and a model's loss.
What you'll be able to do
Spot the common slips of Module 6: chances that don't add up to 1, dependent events multiplied as if independent, P(A | B) swapped with P(B | A), a simulation with no seed, and a variance left without its square root
Find and fix slips in probability code that runs but gives a wrong answer
Mix problems from every lesson of the module, and run a short probability program in your head, line by line
Use the module's formula sheet: each rule, what it says, and its Python line, and see where Module 6 sits in a model
Quick check
Have a guess before you start. A spam filter flags 95% of spam. A friend says: "So a flagged email is 95% likely to be spam." Is that right?
No: it swaps P(flag | spam) for P(spam | flag)
Only if the filter is new
Yes, always
It's right when there are more than 100 emails
Show the answer
No: it swaps P(flag | spam) for P(spam | flag)
95% is P(flagged | spam). P(spam | flagged) also depends on how much email is spam and how often good mail is flagged: Bayes' rule.
Find the slip
Purpose. Five slips cover most of Module 6's mistakes, and each has a quick check:
Chances that don't add up to 1. [0.5, 0.3, 0.1] isn't a distribution. Check: p.sum() must be 1.
Dependent events multiplied. P(A and B) = P(A) × P(B) only when A doesn't change B's chance. Check: is B's chance the same once A has happened? If not, use P(A) × P(B | A).
P(A | B) swapped with P(B | A). They restrict to different groups. Check: say aloud which group is being counted in, and use Bayes' rule to move between them.
No seed.default_rng() with nothing inside gives new numbers every run. Check: run it twice; the answer must not change.
A variance left as a spread. Σ (x − μ)²p is in squared units. Check: the units. A spread in tents² is a variance; take the square root for σ.
Below are ten worked lines, with a comment saying what each should give. Five have a slip.
Try it now
How the checks catch them. Each slip fails a one-line test. p.sum() prints 0.9 for the next-word chances. Asking "is tomorrow's chance the same after a rainy day?" gives 0.7 against 0.4, so the events are dependent. Saying "among the positives, how many are sick?" shows that 0.9 answered a different question, among the sick. Running the simulation twice prints two different answers. And the units of Σ (x − μ)²p come out squared. Predict the next one before you run it.
Quick check
What does this print?
import numpy as np
v = np.array([0.0, 10.0])
p = np.array([0.5, 0.5])
mean = v @ p
var = np.sum((v - mean) ** 2 * p)
print(var, var ** 0.5)
25.0 25.0
25.0 5.0
5.0 25.0
5.0 5.0
Show the answer
25.0 5.0
Both values sit 5 from the mean of 5, so the variance is 5² = 25 and σ, its square root, is 5: the typical distance, in the data's units.
Slips that hide in code. Like the earlier modules', these slips mostly run without an error. rng.choice refuses chances that don't add up to 1, but an expected value v @ p or a sampler by slices (lesson 91) uses them without complaint, so the slip surfaces far from where it was made. A Bayes calculation that forgets one path into P(B) returns a number between 0 and 1 that looks like a probability. The two drills below are slips of exactly this kind: each runs, each gives a confident wrong answer, and one check exposes each.
Quick check
A disease affects 1 in 100 people. A test is positive for 90% of the sick and 5% of the well. Roughly what is P(sick | positive)?
0.9
About 0.15
0.01
0.05
Show the answer
About 0.15
Of 1,000 people: 9 sick test positive, and about 50 of the 990 well do too. 9 of 59 positives are sick: about 0.15.
Drill
Chances that don't add up
This code turns next-word counts into chances, but they add up to more than 1. Find the slip and fix it.
import numpy as np
counts = np.array([6, 3, 1])
p = counts / 6print(p.tolist(), float(np.sum(p)))
In the lesson, your code is checked as you write it.
In your own words
Which check would you run on any list of chances before sampling from it, and why that one first?
Write a sentence or two, then compare it with a good answer in the lesson.
Mixed practice
Purpose. Problems from across Module 6, mixed. Name each one's lesson before you start: counting, chances, and-or, conditionals, Bayes, random variables, expected values, named distributions or covariance. If one stops you, the steps show the way, and the formula sheet in the last part has every rule in one place.
Play computer. The program below runs the module in small: a count of groups, chances from counts and their check, an expected value, a variance and its square root, and two dependent events in a row. Most lines are a spot where one of the first part's slips could creep in, written here the right way. Work out every value in your head before you type it, the way Python prints it. .item() turns NumPy's one-number answers into plain Python numbers, so they print plainly.
Worked example
Worked example: a simulation with its checks built in
Simulate a rainy-day chain, and check the chances add up to 1, the run repeats exactly, and the answer agrees with the multiplication rule.
import numpy as np
p_today = np.array([0.4, 0.6])
print("adds to 1:", float(p_today.sum()) == 1.0)
defsimulate(seed, n):
rng = np.random.default_rng(seed)
both = 0for day inrange(n):
rain = rng.random() < 0.4if rain:
if rng.random() < 0.7:
both = both + 1return both / n
a = simulate(1, 20000)
print("repeats:", a == simulate(1, 20000))
print(round(a, 2), round(0.4 * 0.7, 2))
It runs right here in your browser, with nothing to install.
The first check would catch chances that don't add up; the second, a missing seed, since the same seed must give the same answer. The last line compares the simulation, about 0.28, with the multiplication rule for dependent events, P(rain) × P(rain | rain the day before) = 0.4 × 0.7 = 0.28. Had the code used 0.4 × 0.4 = 0.16, the simulation would have disagreed at once. Three cheap checks catch most probability bugs.
Quick check
What does this print?
import numpy as np
counts = np.array([2, 3, 5])
p = counts / counts.sum()
print(p.tolist(), float(p.sum()))
[0.2, 0.3, 0.5] 10.0
[0.2, 0.3, 0.5] 1.0
[0.4, 0.6, 1.0] 1.0
[2, 3, 5] 10.0
Show the answer
[0.2, 0.3, 0.5] 1.0
Dividing each count by the total, 10, gives chances that add up to exactly 1: a distribution.
Quick check
Two cards are drawn from 10, of which 4 are red, without putting the first back. Which is P(both red)?
4/10 × 3/9
4/10 + 3/9
4/10 × 4/10
4/10
Show the answer
4/10 × 3/9
Without replacement the second draw depends on the first: once a red is gone, 3 reds are left among 9 cards.
Drill
A simulation nobody can check
This function should estimate the chance of rolling a six, and give the same answer every time it runs with the same seed. It doesn't. Find the slip and fix it.
Which row of the sheet so far catches a P(A | B) read as P(B | A)?
Lesson 87: groups
Lesson 89: the "or" rule
Lesson 90: Bayes' rule
Lesson 88: the complement
Show the answer
Lesson 90: Bayes' rule
Bayes' rule is exactly the bridge from P(B | A) to P(A | B), and it shows they differ unless P(A) = P(B).
The rest of the sheet:
lesson
rule
in plain words
in Python
91
Σ pᵢ = 1; P(range) = Σ of its pᵢ
a chance for every value
rng.choice(v, size=n, p=p)
92
E[X] = Σ xᵢpᵢ; Var = Σ (xᵢ − μ)²pᵢ; σ = √Var
balance point; spread, in the data's units
v @ p, ((v - mu) ** 2) @ p, var ** 0.5
93
P(k) = C(n, k) pᵏ(1 − p)ⁿ⁻ᵏ; mean np; z = (x − μ)/σ
yeses in n trials; distance in σ's
math.comb(n, k) * p ** k * (1 - p) ** (n - k)
94
cov = mean of (x − x̄)(y − ȳ); r = cov ÷ (σₓσᵧ)
moving together, from −1 to 1
np.mean(dx * dy), np.corrcoef(x, y)
Each row is also a check: if an answer breaks a row, the slip is in that step.
Where Module 6 lives in a model. Probability is how a model talks about what it doesn't know. It turns scores into chances that add up to 1, one per possible next word, and sampling draws from them. Its loss is an expected value, averaged over a batch. Its precision and recall are conditional chances, and its accuracy needs an error bar, from the binomial. Counting says how attention's work grows with length, and chained chances say how fast many steps that usually work become likely to fail. Its starting weights are random draws, and covariance shows which inputs repeat each other.
Coming up. First, lesson 96 turns this module's formulas around: Bayes' rule solved for the prior, and expected values, variances and z-scores run backward. Then Module 7's chances meet real data. Lesson 97 describes a column of numbers with the same mean and spread, divided by n or n − 1. Lesson 100 picks a model's numbers to make the data as likely as possible, and lessons 104 and 105 turn a model's scores into a distribution like the ones here, then measure how surprised it is by the right answer: the loss that every model in the academy is trained on.
In your own words
Pick the rule on the sheet you're most likely to slip on. What check would you use to catch the slip?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write