A sequence is a list of numbers that follows a rule, and Σ (the Greek capital sigma) is a single symbol for adding many of them. Almost every formula in machine learning has a Σ in it: an average, a loss, a dot product. Once you can read Σ as "a loop that adds", those formulas stop looking like code in a foreign language. This lesson reads and writes Σ, then uses it to measure how wrong a model is. Along the way you'll meet arithmetic and geometric sequences, subscripts, and Π, Σ's cousin for multiplying.
What you'll be able to do
Find the nth term of arithmetic and geometric sequences
Read and write Σ notation, with its index, limits and subscripts, and see the mean as a sum over a count
Turn any Σ into a running total in a Python loop, or into sum()
Compute a model's mean squared error, the Σ at the heart of training
Quick check
Have a guess before you start. What comes next: 2, 5, 8, 11, …?
16
14
22
13
Show the answer
14
Each term adds 3: 11 + 3 = 14. A sequence that adds the same amount each step is called arithmetic.
The math: patterns, and adding them up
Purpose. Describe patterns that go step by step, and add many numbers with one symbol.
A sequence is a list that follows a rule. Two kinds come up most:
Arithmetic: add the same amount d each step. 2, 5, 8, 11 has d = 3. The nth term is a + (n − 1)d, so the 10th is 2 + 9 × 3 = 29.
Geometric: multiply by the same factor r each step. 5, 10, 20, 40 has r = 2. The nth term is a·rⁿ⁻¹, so the 7th is 5 × 2⁶ = 320.
These are lesson 21's straight lines and lesson 24's exponentials, taken one whole step at a time.
Quick check
From an earlier lesson: a value starts at 3 and doubles each step. What is it after 4 steps?
24
96
48
11
Show the answer
48
3 × 2⁴ = 3 × 16 = 48. A geometric sequence is that same doubling, listed term by term: 3, 6, 12, 24, 48.
Σ: one symbol for adding. Instead of writing x₁ + x₂ + x₃ + x₄ + x₅, write
i=1∑5xi
In plain words: start i at 1, and for each i up to 5, add the term xᵢ.
The small i underneath is the index; 1 and 5 are its limits. The subscript in xᵢ names a position, so x₃ is the third term. Watch Σ add the terms 3, 1, 4, 1, 5 one at a time: it's a running total. In the picture, each term is a bar, x₁ to x₅, and the last bar, Σ, is the total so far. Each step lights one term, marked with a plus sign, and the Σ bar grows by exactly that term's height.
Try it now
Then change the terms and press Use these numbers: Σ is always a running total.
Symbols and formula. Σ often wraps a whole expression, and the mean is a Σ divided by a count:
i=1∑4i2=1+4+9+16=30xˉ=n1i=1∑nxi
In plain words: work out the expression for each i and add the results. The mean, x̄ ("x bar"), adds all n values and divides by n.
Σ's cousin Π (capital pi) multiplies instead of adding: Π of 1, 2, 3, 4 is 24. You met Π's job in lesson 25, multiplying probabilities.
Quick check
What is the 10th term of 2, 5, 8, 11, …?
23
29
32
30
Show the answer
29
a + (n − 1)d = 2 + 9 × 3 = 29. From the first term there are 9 steps to the 10th.
Quick check
Which is the mean of 4, 7 and 10, in Σ form?
(1/3) × Π xᵢ
Σ xᵢ = 21
Π xᵢ = 280
(1/3) × Σ xᵢ = 7
Show the answer
(1/3) × Σ xᵢ = 7
Add the three values, 21, and divide by the count, 3: the mean is 7.
In your own words
In your own words: how do you read Σ from i = 1 to n of xᵢ out loud, and what does it do?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: Σ is a loop, or sum()
Purpose. Every Σ can be written as a loop that adds, and simple ones as sum().
xs = [3, 1, 4, 1, 5]
total = 0for x in xs:
total = total + x
print(total, sum(xs))
It runs right here in your browser, with nothing to install.
The loop is Σ written out: start at 0, add each term. sum does the same in one call.
Code flow. Step through the same loop one line at a time. Watch x take each term in turn and total grow by it: the Σ bar from the math part's picture, as a variable.
For a Σ over an expression, compute the term inside the loop:
total = 0for i inrange(1, 5):
total = total + i ** 2print(total)
print(sum(range(1, 101)))
The second line adds 1 to 100 in one call: 5,050.
A named mistake: off by one at the limits. Math's Σ from i = 1 to 4 includes 4, but range(1, 4) stops before 4. To include the top limit n, use range(1, n + 1). And math counts positions from 1 while Python counts from 0: x₁ in math is xs[0] in code.
A running list. Keeping each partial total in a list gives a cumulative sum, useful for things like a running count of signups:
run = []
t = 0for x in [4, 2, 5, 1]:
t = t + x
run.append(t)
print(run)
Worked example
Worked example: a mean, three ways
Find the mean of 4, 7 and 10 with a loop, with sum(), and check it against the Σ formula.
xs = [4, 7, 10]
total = 0for x in xs:
total = total + x
print(total / len(xs))
print(sum(xs) / len(xs))
In the lesson you can change this and run it.
The loop builds Σ xᵢ = 21 step by step, then divides by n = 3, using len so the code works for a list of any length. sum(xs) / len(xs) is the same formula, (1/n) Σ xᵢ, in one line. Both give 7.0. Writing the loop once by hand is worth it: when a Σ has a complicated term inside, the loop is how you'll compute it.
Quick check
What does this print?
total = 0for i inrange(1, 4):
total = total + 2 * i
print(total)
12
20
6
8
Show the answer
12
range(1, 4) gives 1, 2, 3. The terms 2 × i are 2, 4, 6, which add to 12.
Quick check
Which code computes Σ from i = 1 to n of i, including n?
sum(range(n))
sum(range(1, n + 1))
sum(range(1, n))
range(1, n + 1)
Show the answer
sum(range(1, n + 1))
range stops before its second number, so n + 1 includes n itself.
Sums with a shortcut. Some sums have a neat closed form. Adding 1 to n gives n(n + 1)/2: pair the first and last terms, 1 + n, and there are n/2 such pairs. So 1 to 100 is 100 × 101 ÷ 2 = 5,050, as Python printed. A formula like this is a good check on a loop, and a loop is a good check on a formula.
Drill
Two sums and a cumulative list
Fix the code: s1 should be Σ i³ for i = 1 to 4, s2 the mean of xs, and cum the running totals of xs.
s1 = 0for i inrange(1, 4):
s1 = s1 + i ** 3
xs = [6, 1, 8, 5]
s2 = sum(xs) / 3
cum = []
t = 0for x in xs:
t = x
cum.append(t)
print(s1, s2, cum)
In the lesson, your code is checked as you write it.
In your own words
Why do math and Python disagree about where a list starts, and how do you keep them straight?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: almost every formula is a Σ
Purpose. The loss a model is trained on is a Σ.
The most common loss for predicting numbers is the mean squared error:
MSE=n1i=1∑n(y^i−yi)2
In plain words: for each example, square the miss between the prediction ŷᵢ and the answer yᵢ. Add them all up, then divide by how many there are.
xs = [1, 2, 3, 4, 5, 6]
ys = [3, 5, 7, 8, 11, 13]
w = 2
b = 1
total = 0for i inrange(len(xs)):
total = total + (w * xs[i] + b - ys[i]) ** 2print(total, total / len(xs))
The line ŷ = 2x + 1 hits five of the six points exactly and misses one by 1, so the mean squared error is 1 ÷ 6. A perfect fit would score exactly 0; every miss, in either direction, pushes the score up.
The picture. Why "squared"? Draw a miss as the side of a square, and its squared error is the square's area. Below are the six points and the line. Press Next to zoom in on the one miss, at x = 4.
Quick check
What does this print?
preds = [3, 5, 9]
ys = [2, 5, 7]
total = 0for i inrange(len(ys)):
total = total + (preds[i] - ys[i]) ** 2print(total / len(ys))
0.6666666666666666
5.0
1.3333333333333333
1.6666666666666667
Show the answer
1.6666666666666667
The misses are 1, 0 and 2, which square to 1, 0 and 4: a total of 5. Divided by 3, the mean squared error is 1.666….
Reading Σ in papers. Once Σ reads as a loop, much of ML notation opens up. A dot product, Σ wᵢxᵢ, multiplies two lists position by position and adds (lesson 50). Cross-entropy, −Σ yᵢ log pᵢ, adds up surprises (lesson 105). Training averages the loss over a batch of examples: a Σ divided by the batch size. In each case the recipe is the same: find the index, find the term, and add. Lesson 27 gives each term a weight, Σ wᵢxᵢ, to build the averages that smooth a noisy loss.
Why the 1/n? Without it, a model's loss would grow with the size of the data: a thousand examples would always look worse than ten. Dividing by n turns the total into an average error per example, so losses on different data can be compared. Some formulas use the plain total instead, and the best weights come out the same either way, because dividing by a fixed n doesn't move the bottom of the bowl.
Drill
The mean squared error of a line
Fix mse(w, b) so it returns the mean squared error of the line ŷ = w·x + b on xs and ys. Then set err to the error of w = 2, b = 1.
xs = [1, 2, 3, 4, 5, 6]
ys = [3, 5, 7, 8, 11, 13]
defmse(w, b):
total = 0for i inrange(len(xs)):
total = total + (w * xs[i] + b - ys[i])
return total
err = mse(2, 1)
print(err)
In the lesson, your code is checked as you write it.
In the real tools. PyTorch, the library most models are built with, has the whole formula as one call: square each miss, add, divide by n. Here is the line ŷ = 2x + 1 on the same six points:
In your own words
Why does a model's loss use a Σ?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write