A chance changes when you learn something. The chance of rain is one number; the chance of rain given that the sky is black is another. That "given" is conditional probability, and it's everywhere in machine learning: a language model predicts the next word given the words so far, and a spam filter scores a message given its words. A classic trap comes with it: the chance a test is positive given you're sick is not the chance you're sick given the test is positive, and confusing them can be badly wrong. Bayes' rule is the bridge between the two. This lesson builds both from a simple table of counts.
What you'll be able to do
Find P(A | B) from a two-way table: keep only the B column, then take the share
See that P(A | B) and P(B | A) are different numbers, often very different
Use Bayes' rule, drawn as paths on a tree, to turn one into the other, as for a test with false alarms
Read a classifier's results table both ways, precision and recall, in NumPy and with PyTorch
Quick check
Have a guess before you start. Of 100 hikers, 30 carried a map, and 24 of those finished the trail. Of the map carriers, what share finished?
0.24
0.8
0.3
0.72
Show the answer
0.8
Restrict to the 30 map carriers: 24 of 30 finished, 0.8. That's a conditional probability: finished, given a map.
The math: restrict, then take the share
Purpose. Update a chance when something is known.
A sign classifier was tested on 100 signs. The table counts what each sign was (rows) and what it was called (columns):
blank
called tent
called tarp
tent
40
10
tarp
5
45
"Given it was called tent" means: look only at that column, 45 signs. Of those, 40 were tents:
P(A∣B)=P(B)P(A and B)=4540≈0.889
In plain words: shrink the world to the cases where B happened, then ask what share of those have A.
Quick check
From lesson 88: what is P(called tent) in that table?
0.4
0.45
0.9
0.5
Show the answer
0.45
40 + 5 = 45 of the 100 signs were called tent.
Step through conditioning on the "called tent" column: keep it, then turn its counts into shares.
The picture. The table is drawn as areas: a bar as wide as all 100 signs, with each column of the table on its own line and each piece as wide as its count. The empty part of each line is room for the other column's signs. The first step keeps one column and fades the other. Then the kept column gets a bar of its own: its signs are now the whole, and each piece's share of that bar is a conditional chance.
Try it now
Then set the column to keep to 1: given "called tarp", the shares are 10/55 and 45/55.
Symbols and formula. The two directions use different totals:
In plain words: the first restricts to a column (calls); the second restricts to a row (actual tents).
P(A | B) is not P(B | A). Here they're close, 0.889 and 0.8, but they can be wildly different. Of all cows, nearly all have four legs; of all four-legged animals, few are cows. The order of the bar matters.
Quick check
In the sign table, what is P(called tarp | tarp)?
0.5
0.818
0.45
0.9
Show the answer
0.9
Restrict to the tarp row, 50 signs: 45 were called tarp, 0.9.
Bayes' rule. Often you know P(B | A) and want P(A | B). Draw it as paths: multiply down each branch to the evidence, add the branches for the total, and take one branch's share:
P(A∣B)=P(B)P(B∣A)P(A),P(B)=P(B∣A)P(A)+P(B∣not A)P(not A)
In plain words: the share of all the ways to see B that came through A.
A test catches 90% of a condition that 1% of people have, but also flags 5% of healthy people. Step through it as a tree. From one person, the first branches say sick (0.01) or healthy (0.99); the second say what the test shows. Each step multiplies down one path, and the last adds the two paths to a positive and takes the sick path's share.
Most positives are false alarms, because healthy people vastly outnumber sick ones. A good test, used to look for something rare, still gives a positive that is usually wrong.
In your own words
In your own words: why is P(sick | positive) so much lower than P(positive | sick) for a rare condition?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: a table as an array
Purpose. Store counts as a 2-D array, then get every conditional by dividing.
It runs right here in your browser, with nothing to install.
T.sum(axis=0) adds down each column (lesson 64); axis=1 adds across each row. T / cols divides every column by its own total, by broadcasting: each column now holds P(row | that column). Dividing by the row totals, reshaped into a column so they line up with the rows, gives P(column | row) instead.
Code flow. Step through dividing the table by its row totals, one row at a time. np.zeros((2, 2)) makes a 2 × 2 table of zeros for the loop to fill with shares. Each pass of the loop takes one row's total and divides that row by it, so the row becomes P(called | what it was). The last line does all of it at once, the broadcasting way, and prints the same table.
A named mistake: the wrong total. Dividing a cell by the grand total gives P(A and B), not a conditional. Dividing by its row total gives P(column | row); by its column total, P(row | column). Before dividing, say the condition aloud: "given called tent" names a column, so divide by that column's total.
Every conditional at once. The two divided tables above hold all eight conditionals of the 2 × 2 table, each column (or row) adding to 1. That's a quick check on the arithmetic: if a conditioned column doesn't add to 1, the wrong total was used.
Worked example
Worked example: every conditional in a 2 × 3 table
A camp survey: rows are "liked it" and "didn't", columns are three trails. Find P(liked | trail) for each trail.
Each trail had 35, 35 and 30 hikers. The share who liked each: 30/35 ≈ 0.857, 10/35 ≈ 0.286 and 10/30 ≈ 0.333. The last line checks the arithmetic: every conditioned column adds to 1. Comparing conditionals, not raw counts, is fair even when the trails had different numbers of hikers.
Quick check
What does this print?
import numpy as np
T = np.array([[8, 2],
[2, 8]])
print((T / T.sum(axis=0)).tolist())
[[8, 2], [2, 8]]
[[0.4, 0.1], [0.1, 0.4]]
[[0.5, 0.5], [0.5, 0.5]]
[[0.8, 0.2], [0.2, 0.8]]
Show the answer
[[0.8, 0.2], [0.2, 0.8]]
Each column totals 10, so each cell is divided by 10: the shares within each column.
Drill
Bayes' rule as a function
Fix bayes(prior, hit, false_alarm) so it returns P(A | positive) for a test with P(positive | A) = hit and P(positive | not A) = false_alarm. Then set p1 for prior 0.01, hit 0.9, false alarm 0.05, and p2 for prior 0.3 with the same test, both rounded to 4 places.
In the lesson, your code is checked as you write it.
In your own words
The same test gives P(A | positive) of 0.15 when A is rare and 0.89 when it's common. Why?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: predictions are conditional
Purpose. Models predict one thing given another.
A language model gives P(next word | the words so far); a spam filter gives P(spam | the message's words). A classifier's results table, like the sign table, gives two numbers people care about, and they're conditionals in opposite directions:
For "tent", precision is 0.8889 (column) and recall 0.8 (row). A classifier can be high on one and low on the other, which is why both are reported. The picture lights the cells each one divides by.
Quick check
A classifier calls 10 signs "tent", and all 10 are tents, but it misses 40 other tents. What are its precision and recall for tent?
Precision 1.0, recall 0.2
Both 0.2
Precision 0.2, recall 1.0
Both 1.0
Show the answer
Precision 1.0, recall 0.2
All its tent calls were right: precision 10/10. It found only 10 of the 50 tents: recall 10/50 = 0.2.
Naive Bayes. One of the oldest text classifiers runs Bayes' rule directly: P(spam | words) is built from P(words | spam), counted from labeled examples, and the prior P(spam). It assumes the words are independent given the class, which is false but works surprisingly well. This lesson's paths are its whole calculation.
Precision and recall pull against each other. They share the hits on top and differ only in the total below, exactly like P(A | B) and P(B | A). A model can raise its recall by calling "tent" more often: it catches more real tents, but it also calls more tarps "tent", which lowers its precision.
In PyTorch. The same two divisions on a tensor. T.sum(dim=1, keepdim=True) adds across each row and keeps the totals standing as a column, (2, 1), so they line up with the rows, the job .reshape(2, 1) did in the Python part. dim=0 adds down each column instead.
import torch
T = torch.tensor([[40.0, 10.0], [5.0, 45.0]])
by_row = T / T.sum(dim=1, keepdim=True) # P(called | what it was)
by_col = T / T.sum(dim=0, keepdim=True) # P(what it was | called)print(by_row)
print(round(by_col[0, 0].item(), 4), round(by_row[0, 0].item(), 4))
The first row of by_row splits the 50 real tents: 0.8 were called tent (the recall) and 0.2 called tarp. The last line prints 0.8889 and 0.8, the precision and recall the NumPy cell printed. Evaluation code does exactly this to a model's results table, one row and one column per class.
Quick check
What does this print? (Rows: actual tent, tarp. Columns: called tent, tarp.)
Precision: 8 of the 12 called tent, 0.667. Recall: 8 of the 10 real tents, 0.8.
Drill
Precision and recall from a table
For the classifier's table below (rows: actual tent, tarp; columns: called tent, tarp), set precision and recall for tent, and tarp_precision for tarp, each rounded to 4 places.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write