Index Ai Academy
Foundation · Foundation mathTransposition of formulas

Statistics and information: n, z, logits and bits

About 40 minLooking ahead: some prerequisites aren't done yet

Why this matters

Statistics usually runs forward: from the data to an average, an error bar or a chance. Planning runs it backward. How many test questions give a ±2-point error bar? What input tips a classifier to 90%? What temperature turns these scores into those chances? What chance does a loss of 0.7 stand for? Each answer is a formula from the statistics module with a different letter made the subject. The moves are this subcourse's: a square undoes a root, a log undoes an exponential, and a power undoes a log.

  • Turn the mean around for a missing value or for n, the standard error and a margin around for the sample size (rounded up), and a margin around for z
  • Run the regression line backward: x̄ from the intercept, and the x that gives a target prediction
  • Undo the sigmoid, softmax and temperature: the input for a target chance, the score gap for a ratio of chances, and T
  • Turn bits and losses into chances: p = 2^(−s), n = 2^H, p = e^(−L), and perplexity
  • State the conditions: n rounded up, p strictly between 0 and 1, a slope that isn't 0, and logs only of numbers above 0
Quick check
Have a guess before you start. An average of 25 people has a standard error of 2. About how many people bring it down to 1?

The math: turning statistics around

Purpose. Solve the statistics module's formulas for a different letter: n, a missing value, z or an input.

The mean. x̄ = Σx/n: add the values, then divide by how many there are. Multiply both sides by n and you get the total: Σx = n·x̄. That one move frees two letters.

  • A missing value is the total minus the values you know. Four shuttle waits average 8 minutes, and three of them are 6, 9 and 7. The total is 4 × 8 = 32, so the lost one is 32 − 22 = 10.
  • n is the total divided by the mean: n = Σx/x̄. The trail club walked 96 km this month at 12 km a hike, so 96 ÷ 12 = 8 hikes. The condition: x̄ can't be 0.

The standard error. SE = σ/√n is how much the mean of n values wobbles from sample to sample (lesson 98). Here σ is the spread of single values. n gets two layers: a square root, then σ divided by it. Peel them in reverse. Swap √n and SE, so σ ÷ SE = √n. Then square both sides: n = (σ/SE)².