Review: find the slip (statistics and information)
Why this matters
Module 7's slips are the ones that reach reports. A mean pulled up by one slow request goes on a dashboard as "typical". A loss that hits a probability of exactly 0 prints inf, and a night of training is lost. A softmax without its exponential still adds up to 1, and quietly gives the wrong chances. A sample size planned on "twice the data, half the wobble" comes out four times too small. A model's training error is reported as what users should expect. Recall is reported where precision was meant, a two-point lead between models is announced before anyone asks whether it's luck, and a KL divergence is computed the wrong way around. Each runs without complaint. This review gathers the eight, each with the check that catches it, mixes problems from all of Module 7, and collects every rule on one sheet, ready for Module 8.
- Spot the common slips of Module 7: a mean pulled by an outlier, a log of zero, softmax without the exponential, twice the data expected to halve the wobble, training error reported as test error, precision and recall swapped, a lucky win taken as real, and KL turned around
- Find and fix slips in statistics code that runs but gives a wrong answer
- Mix problems from every lesson of the module, and run a short statistics program in your head, line by line
- Use the module's formula sheet: each rule, what it says, and its Python line, and see where Module 7 sits in a model
Find the slip
Purpose. Eight slips cover most of Module 7's mistakes, and each has a quick check. The first five come from describing data, samples, regression, likelihood and softmax:
- A mean pulled by an outlier. One 40-second request makes the "typical" time 10 seconds. Check: compare the mean with the median; a big gap means a lopsided column.
- A log of zero. A probability that rounds to exactly 0 makes −log p infinite. Check: is the loss finite? Clip probabilities into [ε, 1 − ε] before the log.
- Softmax without the exponential.
z / z.sum()adds up to 1 but isn't softmax, and breaks on negative scores. Check: a score of 0 must still get a positive share. - Twice the data for half the wobble. The standard error divides by √n, so halving it takes four times the data. Check: σ/√(2n) is σ/√n divided by only 1.41.
- Training error reported as test error. Check: which rows was the error computed on? Only held-back rows count.
Put numbers on the fourth slip before reading on.