Mean, variance and spread

Same average, very different ride

Picture two ways to get to work. Both take 30 minutes on average. The bus takes 28 to 32 minutes every day. The car takes 15 minutes on a clear day and 60 in a jam. The averages match, but only one lets you plan your morning.

The average alone can't tell these apart. You need a second number that measures spread: how far single results tend to land from the average. The recipe has three steps.

1. Find how far each value is from the mean. This gap is called the deviation.
2. Square each gap, which means multiply it by itself. This makes every gap positive. Without it, gaps above and below the mean would cancel each other out.
3. Take the weighted average of those squares, using the same chances as before.

The result is the variance. Because of the squaring, it's in odd units like "squared dollars". So you usually take its square root: the number that, times itself, gives the variance. For example, the square root of 25 is 5. That square root is the standard deviation. It's back in the original units, like dollars or minutes. Think of it as "the typical distance from the average".

Two more Greek letters help here. $\mu$ (mu, said "mew") is the mean. $\sigma$ (a small sigma) is the standard deviation. It's a different letter from the big $\sum$ that means "add up".

In plain words: the variance is the average squared distance from the mean. The standard deviation is its square root, so it's in the same units as the data.

A worked example: a coin game pays $0 on tails and $10 on heads. The mean is $5. Both outcomes are $5 away from it. Squared, each gap is 25. The weighted average is 0.5 × 25 + 0.5 × 25 = 25. So the variance is 25, and the standard deviation is the square root of 25, which is $5. That matches your gut: every result is $5 from the average.

values = [0, 10]
probs = [0.5, 0.5]
mean = sum(v * p for v, p in zip(values, probs))
var = sum((v - mean) ** 2 * p for v, p in zip(values, probs))
print(mean, var, var ** 0.5)

It prints 5.0 25.0 5.0: the mean, the variance and the standard deviation. In Python, ** 0.5 takes a square root.

Now a real choice. You have two campaign options. "Steady" earns $9,000, $10,000 or $11,000, with chances 25%, 50% and 25%. "Moonshot" earns $0 (60% of the time), $10,000 (25%) or $40,000 (15%). The code below does the same three steps with NumPy. NumPy multiplies arrays item by item, so d["values"] * d["probs"] multiplies each value by its chance in one go.

import numpy as np

# Two campaign options: revenue outcomes and their probabilities
steady = {"values": np.array([9_000, 10_000, 11_000]), "probs": np.array([0.25, 0.5, 0.25])}
moonshot = {"values": np.array([0, 10_000, 40_000]), "probs": np.array([0.6, 0.25, 0.15])}

for name, d in [("steady", steady), ("moonshot", moonshot)]:
    mean = np.sum(d["values"] * d["probs"])
    var = np.sum((d["values"] - mean) ** 2 * d["probs"])
    print(f"{name:<9} expected = {mean:>8,.0f}   std = {np.sqrt(var):>8,.0f}")

Steady expects $10,000, with a standard deviation of about $707. Moonshot expects $8,500. That's close enough to be tempting. But its standard deviation is about $13,883, nearly twenty times larger. Most of the time it earns nothing. Whether that's acceptable is a business decision. The numbers make the trade-off plain to see.

def mean_and_sd(values, probs):
    mean = sum(v * p for v, p in zip(values, probs))
    var = sum((v - mean) ** 2 * p for v, p in zip(values, probs))
    return mean, var ** 0.5

probs = [0.25, 0.5, 0.25]
for name, values in [("A", [4, 5, 6]), ("B", [-3, 5, 13])]:
    mean, sd = mean_and_sd(values, probs)
    print(name, "mean", mean, "sd", round(sd, 2))

From real data: sample mean and variance

So far you knew the chances in advance. Usually you don't. Instead you have a list of results, like last week's daily sales or the reply times of 50 support emails. Then every observation gets equal weight.

Call the number of observations $n$. Write each observation as $x_i$. The small $i$ counts through the list: $x_1$, $x_2$, and so on. The mean of the data is written $\bar{x}$, said "x bar". The sample variance is written $s^2$.

In plain words: the mean is the ordinary average: add the values and divide by how many there are. The sample variance adds up the squared distances from that mean. Then it divides by one less than the count, $n - 1$, instead of by $n$.

Why one less? Your mean was worked out from the same data. No other number sits closer to those points overall. The true mean, which you don't know, is almost always a bit farther away. So the squared distances from your mean come out a little too small. Dividing by a slightly smaller number nudges the answer back up.

A worked example with three values: 2, 4 and 6. The mean is 4. The distances from it are −2, 0 and 2. Squared, they're 4, 0 and 4, which add to 8. Divide by $n - 1 = 2$ and the variance is

1. The standard deviation is the square root of 4, which is 2. Dividing by $n = 3$ instead

would give about 2.667.

In NumPy, you choose with ddof, the number taken off $n$. Use np.var(x, ddof=1) and np.std(x, ddof=1) for the $n - 1$ version. The default, ddof=0, divides by $n$.

import numpy as np

data = np.array([2, 4, 6])
print(data.mean())
print(np.var(data, ddof=1), np.std(data, ddof=1))
print(round(float(np.var(data)), 3))

It prints 4.0, then 4.0 2.0, then 2.667. With large samples the difference is negligible. Dividing the sum of 1,000 squared distances by 999 or by 1,000 gives almost the same answer.

Shortcuts: shifting and scaling

Picture a class photo where every kid stands on the same 20 cm step. The average height goes up by 20 cm. But the gap between the tallest and shortest kid doesn't change. Adding the same amount to everything moves the average and leaves the spread alone.

Now picture converting prices from dollars to cents: multiply everything by 100. The average becomes 100 times bigger. So does every distance from it. Multiplying stretches the spread too.

You can write both changes as $aX + b$. Here $a$ is the number you multiply by. And $b$ is the number you add afterwards.

In plain words: the new mean is the old mean, times $a$, plus $b$. Adding $b$ doesn't touch the spread. Multiplying by $a$ multiplies the variance by $a$ squared. So the standard deviation is multiplied by $a$, ignoring any minus sign.

A worked example: two values, 4 and 6. Their mean is 5 and their standard deviation is 1. Double them and add 10, and you get 18 and 22. The new mean is 2 × 5 + 10 = 20. The new standard deviation is 2 × 1 = 2. The variance went from 1 to 4, which is 2 squared times 1.

import numpy as np

x = np.array([4.0, 6.0])
y = 2 * x + 10
print(x.mean(), x.std())
print(y.mean(), y.std())

It prints 5.0 1.0 and then 20.0 2.0.

Adding random amounts together

Now add two random amounts, like the sales of two separate campaigns. Means always add. If one averages $3,000 and the other $5,000, together they average $8,000. This holds even when the two are linked.

Spread works differently. Variances add, but only when the two are independent: one doesn't affect the other. Standard deviations don't add.

Call the first amount $X$. Call the second one $Y$, like the second campaign's sales.

In plain words: the average of a total is the total of the averages. For independent amounts, the variance of a total is the total of the variances. Add variances, then take the square root. Never add standard deviations.

A worked example: two independent campaigns each have a standard deviation of $3,000. Work in thousands to keep the numbers small. Each variance is 3 × 3 = 9. Together that's 9 + 9 = 18. The square root of 18 is about 4.24. So the total's standard deviation is about $4,240, not $6,000. It's smaller because good and bad weeks partly cancel. When one campaign has a bad week, the other often doesn't.

sd_one = 3
var_total = sd_one ** 2 + sd_one ** 2
print(var_total, round(var_total ** 0.5, 2))

It prints 18 4.24.

This rule explains why spreading your bets works, which is called diversification. Take $n$ independent campaigns with the same spread. The expected total grows $n$ times. The standard deviation grows only by the square root of $n$, written $\sqrt{n}$. With 4 campaigns, the total is 4 times bigger but only 2 times as spread out. With 100 campaigns, it's 100 times bigger but only 10 times as spread out. Compared with its size, the total gets steadier. The same $\sqrt{n}$ shows up in the next two lessons. It's the reason bigger samples give more precise estimates.

The code below checks this by simulation. rng.normal(10_000, 4_000, ...) draws random campaign results that average 10,000 with a standard deviation of 4,000. (That bell-shaped "normal" distribution gets its own lesson soon.) It draws 100,000 single campaigns, and 100,000 groups of twenty. The std/mean column compares the spread to the size.

import numpy as np

rng = np.random.default_rng(1)
one = rng.normal(10_000, 4_000, size=100_000)                      # one campaign
twenty = rng.normal(10_000, 4_000, size=(100_000, 20)).sum(axis=1)  # twenty independent ones
print("one:    mean", round(one.mean()), " std", round(one.std()), " std/mean", round(one.std() / one.mean(), 3))
print("twenty: mean", round(twenty.mean()), " std", round(twenty.std()), " std/mean", round(twenty.std() / twenty.mean(), 3))

For one campaign, the spread is about 40% of the mean (0.399). For twenty, the mean is about 200,000 and the standard deviation about 17,900. That's only about 9% of the mean (0.089). The total is 20 times bigger, but the spread is only about 4.5 times bigger. That matches $\sqrt{20}$, which is about 4.47.