Lesson 92 summarized a distribution by its expected value and spread. Real data comes as a column of numbers instead: nine days of tent rentals, ten thousand response times, a million house prices. Nobody reads the whole column, so a few numbers have to stand in for it: where it sits, how widely it spreads, and what its edges look like. Choosing the right few numbers is a skill. One very large value can drag the average far from anything typical, and a feature measured in thousands can drown out one measured in tenths, so a model trained on raw columns may barely learn at all. This lesson builds each summary, shows when each one misleads, and ends with the two jobs it does every day in machine learning: putting features on the same scale, and reporting how fast a system is with percentiles rather than an average.
What you'll be able to do
Summarize a column of numbers by its center (mean, median, mode) and its spread (range, variance, standard deviation, percentiles), and read all three off a histogram
See why deviations are squared, what an outlier does to each summary, and the difference between dividing by n and by n − 1
Compute each summary with a loop, then with np.mean, np.median, np.std and np.percentile, and draw the column with Matplotlib
Standardize features so training works, in NumPy and PyTorch, and report response times as percentiles
Quick check
Have a guess before you start. Nine days of tent rentals average 14. On a tenth day a school group rents 60 tents. What happens to the average?
It stays at 14
It jumps well above anything typical
It rises by a tiny amount
It falls
Show the answer
It jumps well above anything typical
One huge value moves the mean a lot: from 14 to 18.6 here, above every one of the ordinary days.
The math: a column in a few numbers
Purpose. Say where a column of numbers sits and how widely it spreads, in a few numbers.
The center. Nine days of rentals: 12, 15, 11, 14, 15, 15, 17, 16, 11. Three numbers describe their middle:
The mean adds them and divides by how many: 126 ÷ 9 = 14.
The median sorts them (11, 11, 12, 14, 15, 15, 15, 16, 17) and takes the middle one: 15. With an even count, it's the average of the two middle values.
The mode is the most common value: 15, which came up three times.
The mean is lesson 92's expected value with every day weighted equally, 1/9 each. A weighted mean gives some values more say than others, and it comes back in lesson 113, where attention averages words by weights.
Quick check
From lesson 92: if each of the nine days gets probability 1/9, what does the expected value Σ xᵢpᵢ become?
The largest value
The median
The sum of the values
The plain mean, sum ÷ 9
Show the answer
The plain mean, sum ÷ 9
Σ xᵢ × 1/9 = (Σ xᵢ)/9: the plain average is an expected value with equal weights.
The spread, built from a failure. The range, largest minus smallest, is 17 − 11 = 6, but it only looks at the two edges. To use every value, measure each one's distance from the mean, its deviation. The obvious next move fails: the deviations always add up to 0, the positives exactly canceling the negatives, so their average is always 0. Squaring them first removes the signs.
The picture. Step through it for the nine days, as boxes of numbers and as a drawing. The drawing is a histogram: one block per day, stacked over the number of tents that day rented, so the three 15s make the tallest pile. The mean appears as the line the blocks balance on, then each block shows its deviation and its square, and the last step shades one standard deviation either side of the mean and adds the median.
Try it now
Then change the 17 in values to 60 and press Use these numbers. The mean line leaves the pile of ordinary days behind, the band grows wide enough to hold every ordinary day, and the median stays put.
Symbols and formula. For n values x₁ to xₙ:
xˉ=n1i∑xis2=n1i∑(xi−xˉ)2z=sx−xˉ
In plain words: the mean is the balance point, the variance the average squared distance from it, s its square root back in the data's units, and z how many s's a value sits from the mean.
For the nine days: s² = 38 ÷ 9 = 4.22 and s = 2.05 tents. The 17-tent day has z = (17 − 14) ÷ 2.05 = 1.46: about one and a half standard deviations above a typical day. Percentiles split the sorted column: the 50th is the median, and the 95th is the value 95% of the column sits below.
Outliers. Add the 60-tent school day. The mean jumps from 14 to 18.6, above every ordinary day, and s jumps from 2.05 to 13.9. The median barely notices: 15, the same as before. That's why house prices and incomes are reported as medians.
Quick check
What is the median of 3, 9, 4, 7?
5.5
4
5.75
7
Show the answer
5.5
Sorted: 3, 4, 7, 9. With an even count, average the two middle values: (4 + 7) ÷ 2 = 5.5.
In your own words
In your own words: why does an outlier move the mean so much more than the median?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: loops, then one call each
Purpose. Compute every summary, first by hand-written loops, then with NumPy's one-liners.
import numpy as np
r = [12, 15, 11, 14, 15, 15, 17, 16, 11]
total = 0for v in r:
total = total + v
mean = total / len(r)
squares = 0for v in r:
squares = squares + (v - mean) ** 2print(mean, round((squares / len(r)) ** 0.5, 4))
x = np.array(r, dtype=float)
print(x.mean(), np.median(x), round(float(x.std()), 4), np.percentile(x, 50))
It runs right here in your browser, with nothing to install.
The loop and the one-liners agree: mean 14, standard deviation 2.0548, and the 50th percentile is the median, 15.
Code flow. Step through the two loops one line at a time. The first adds up the days for the mean; the second adds up the squared deviations, the same squares the blocks showed in the math part's picture.
A named mistake: n or n − 1.np.std(x) divides by n. Many other tools, and np.std(x, ddof=1), divide by n − 1 instead, which corrects a small underestimate when the column is a sample from something bigger: the sample's own mean sits a little closer to its values than the true mean would. For the nine days the two give 2.05 and 2.18. Neither is wrong, but mixing them makes hand checks fail, so say which one you used. This module divides by n unless it says otherwise.
Worked example
Worked example: summaries with and without an outlier
Summarize the nine days, add the 60-tent day, and summarize again.
One extra day moves the mean by 4.6 tents and multiplies the standard deviation by almost seven, from 2.05 to 13.94, while the median doesn't move at all. Neither summary is wrong; they answer different questions. The mean answers "how many tents in all, shared out per day?", useful for ordering stock. The median answers "what does a typical day look like?", which the school group shouldn't change. The last line shows the n − 1 version for comparison.
Quick check
What does this print?
import numpy as np
x = np.array([2.0, 4.0, 4.0, 4.0, 5.0, 5.0, 7.0, 9.0])
print(x.mean(), x.std(), np.median(x))
5.0 2.0 4.0
5.0 4.0 4.5
4.5 2.0 5.0
5.0 2.0 4.5
Show the answer
5.0 2.0 4.5
The mean is 40 ÷ 8 = 5; the squared deviations add to 32, so the variance is 4 and s = 2; the median averages the middle pair, 4 and 5.
Draw it. A histogram of the nine days, with the mean and the median marked. plt.hist sorts the values into bins and draws a bar for each; the bin edges at 10.5, 11.5 and so on center each bar on a whole number of tents. plt.axvline draws a vertical line: solid for the mean, dashed for the median.
import numpy as np
import matplotlib.pyplot as plt
r = np.array([12, 15, 11, 14, 15, 15, 17, 16, 11])
plt.hist(r, bins=np.arange(10.5, 18))
plt.axvline(r.mean(), color="black")
plt.axvline(np.median(r), linestyle="--")
plt.xlabel("tents rented")
plt.ylabel("days")
plt.show()
Quick check
Read the plot: the solid line, the mean, sits left of the dashed median. Why?
The mean leaves out the three 15s
The low days sit further below the middle
More of the days rented fewer than 14 tents
The median is always bigger than the mean
Show the answer
The low days sit further below the middle
The mean is the balance point, so distance counts. The 11, 11 and 12 sit 4, 4 and 3 tents below the median, while the high days, 16 and 17, sit only 1 and 2 above it, so the mean is pulled left.
Drill
Standardize a column
Fix standardize(x) so it returns each value's z-score: subtract the mean, then divide by the standard deviation. Then set z for the temperatures, and z_mean and z_sd to z's mean and standard deviation, each rounded to 6 places.
import numpy as np
defstandardize(x):
return x - x.mean()
t = np.array([20.0, 25.0, 30.0, 22.0, 28.0])
z = standardize(t)
z_mean = round(float(z.mean()), 6)
z_sd = round(float(z.std()), 6)
print(z.round(3).tolist(), z_mean, z_sd)
In the lesson, your code is checked as you write it.
In your own words
Why does a standardized column always have mean 0 and standard deviation 1, whatever the original numbers were?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: scale before training, percentiles for speed
Purpose. Two everyday uses: putting features on the same scale, and reporting how fast a system is.
Scale.Lesson 79's training loop can fit rentals from temperature. Here it runs for 100 steps on five days, first on the raw temperatures, around 25, and then on the same temperatures standardized:
import numpy as np
t = np.array([20.0, 25.0, 30.0, 22.0, 28.0]) # temperature
y = np.array([30.0, 45.0, 60.0, 40.0, 55.0]) # rentalsdeftrain(x, lr):
w, b = 0.0, 0.0for step inrange(100):
err = w * x + b - y
w = w - lr * 2 * np.mean(err * x)
b = b - lr * 2 * np.mean(err)
returnfloat(np.mean((w * x + b - y) ** 2))
z = (t - t.mean()) / t.std()
print(round(train(t, 0.001), 2), train(t, 0.002))
print(round(train(z, 0.1), 2), round(t.mean(), 2), round(t.std(), 2))
On raw temperatures the loss bowl is steep in w and shallow in b, so a learning rate small enough to stay stable in w, 0.001, crawls in b: after 100 steps the loss is still 16.18. Double it to 0.002 and training blows up, to a loss past 10⁴¹. Standardize the temperatures (mean 25, s 3.69) and the bowl turns round: a learning rate of 0.1 reaches 2.16 in the same 100 steps. With many features the effect is stronger, because a feature in the thousands would drown out one in tenths. Real pipelines standardize every column with the training set's own mean and s, and reuse those numbers on new data.
Quick check
Temperature runs from 15 to 35 and income from 20,000 to 90,000. Why standardize both before training?
Standardizing makes the model more accurate on its own
To remove the outliers
To make every value positive
So neither feature dominates just because of its units
Show the answer
So neither feature dominates just because of its units
On raw scales, income's huge numbers set the shape of the loss bowl. After standardizing, both features range over similar numbers, so one learning rate suits both.
Percentiles. Response times are lopsided: most requests are quick and a few are slow. Services report the median and the 95th or 99th percentile, "p95", because users feel the slow tail that an average hides. Here are ten times in milliseconds, one of them a slow 400. Predict their mean, median and p95 before you check.
The one slow request pulls the mean to 137.4; the median is 107.5. The 95th percentile falls between 130 and 400, at 278.5.
Drill
Standardize a table, report a percentile
X has one row per day and one column per feature (temperature, rentals). Set Z to X standardized column by column (use axis=0), ok to whether every column of Z has mean 0 and standard deviation 1 (round both to 6 places), and p95 to the 95th percentile of the response times, rounded to 1 place.
import numpy as np
X = np.array([[20, 30], [25, 45], [30, 60], [22, 40], [28, 55]], dtype=float)
ms = np.array([120, 95, 110, 400, 105, 98, 130, 102, 115, 99], dtype=float)
Z = X
ok = False
p95 = 0print(Z.round(2).tolist(), ok, p95)
In the lesson, your code is checked as you write it.
In PyTorch. The same table, standardized with tensors: x.mean(dim=0) and x.std(dim=0) give one number per column, as axis=0 does in NumPy. Watch the standard deviations, though. PyTorch's std divides by n − 1 unless you pass correction=0, which divides by n, as NumPy does. torch.quantile gives percentiles, with the percentile written as a share: 0.95 for p95.
With correction=0, Z is exactly the table your drill printed, and p95 is 278.5 again. Two traps show up. The default std, 4.1231 and 11.9373, is the n − 1 kind, so z-scores made with it come out a little smaller than NumPy's. And torch.median gives 105.0, not 107.5: with an even count, PyTorch returns the lower of the two middle values instead of their average. torch.quantile(ms, 0.5) gives 107.5, the median NumPy reports.
In the real tools, scikit-learn does the standardizing as a reusable step. fit learns each column's mean and standard deviation from the training data, and transform applies them to any rows, new days included:
In your own words
A dashboard shows a service's average response time as 137 ms. What would you ask for instead, and why?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write