Reading math notation as code

Σ means "add them all up"

Σ is the Greek capital letter S, for sum. The full form names a counter and its range:

the sum of xᵢ, for i from 1 to n

is read "add up x-sub-i as i goes from 1 to n". The small letter under the Σ is the loop variable, and what follows the Σ is what you add on each pass. In Python it's a loop with a running total, or simply sum():

x = [4, 1, 5]
total = 0
for i in range(len(x)):
    total += x[i]
print(total, sum(x))

What follows the Σ can be any expression: Σ xᵢ² adds up the squares, and Σ (yᵢ − 3) subtracts 3 from each value before adding. Translate the expression for one item, then wrap it in the loop.

Two variations are common. A Σ inside another Σ is a loop inside a loop: Σᵢ Σⱼ Wᵢⱼ adds every cell of a table. And a Σ can run over a collection instead of a counter: "the sum over x in S" means for x in S, which is often the cleaner way to write it in Python too.

x = [2, 3, 4]
total = 0
for i in range(len(x)):
    total += x[i] ** 2
print(total)

Subscripts are positions

A small letter or number written low after a name is a subscript: xᵢ is "the i-th x", and x₂ is the second one. In code, that's indexing: x[i]. Two subscripts mean a table: Wᵢⱼ is row i, column j, which is W[i][j].

One trap: math usually counts from 1, and Python counts from 0. On paper, x₁ is the first value; in Python, it's x[0]. When you translate, decide once which way you're counting and adjust every index together:

x = [10, 20, 30]
print(x[0])         # x₁ on paper: the first value
W = [[1, 2], [3, 4]]
print(W[1][0])      # W₂₁ on paper: row 2, column 1

Subscripts like xᵢ₋₁ mean "the one before": x[i - 1]. That's how formulas describe things like the change from one day to the next, xᵢ − xᵢ₋₁.

Raised numbers need a second look. x² is a power, but x⁽ⁱ⁾, with brackets around the raised letter, usually labels the i-th example in a dataset rather than raising anything to a power. When in doubt, the surrounding text says which is meant.

Hats, bars, stars and Greek letters

Small marks on a letter change its meaning. A hat, ŷ ("y-hat"), means an estimate or a prediction: y is the true value and ŷ is what the model guessed. A bar, x̄ ("x-bar"), means the mean. A star, θ*, usually means the best value of something.

Greek letters are just names, and the same ones turn up again and again: μ (mu) for a mean, σ (sigma) for a standard deviation, α (alpha) for a learning rate, λ (lambda) for how strongly to regularize, and θ (theta) for a model's parameters, all its weights at once.

y = [3, 5, 2]
y_hat = [2.5, 5, 3]          # ŷ: the model's predictions
y_bar = sum(y) / len(y)      # ȳ: the mean of y, written with a bar
print(y_bar)
print([y[i] - y_hat[i] for i in range(len(y))])   # yᵢ − ŷᵢ, each error

In code you can't write the marks, so programmers spell them out: y_hat, x_bar, theta_best. Spelling them out consistently makes your code read like the formula it came from.

y = [3, 5, 2, 8]          # the true values
y_hat = [2.5, 5, 3, 7]    # the model's predictions, ŷ
n = len(y)

mse = 0

print(mse)

Functions and arrows

f(x) = 2x + 1 is a function: put x in, get 2x + 1 out. In Python it's a def with one line. A line like f: ℝ → ℝ just says what goes in and what comes out (ℝ means "any real number"), the way a type hint does.

argmax is worth knowing early: argmax f(x) is the input that makes f biggest, not the biggest value itself. A classifier's prediction is the argmax of its scores, the class with the highest score. A few more symbols complete the everyday set: ∈ means "is in" (Python's in), ∝ means "is proportional to", and := or ≜ mean "is defined as":

def f(x):
    return 2 * x + 1

print(f(3))
scores = {"cat": 0.2, "dog": 0.7, "bird": 0.1}
print(max(scores, key=scores.get))      # argmax: the class, not the score

Reading a whole formula

A formula you've never seen becomes manageable in four steps:

1. Name every symbol. What's a list, what's a single number, what's the counter?
2. Find the loops. Each Σ or Π is a loop; nested ones are nested loops.
3. Find what's done to each item, inside the loop, and what's done after it.
4. Write it, then test it on tiny numbers you can check in your head.

Here's softmax, which you'll meet properly later: softmax(z)ᵢ = e^(zᵢ) / Σⱼ e^(zⱼ). Name the parts: z is a list of scores, e^ is math.exp, and the Σ over j adds up every score's exponential. So each output is one score's exponential divided by the total of all of them:

import math
z = [2.0, 1.0, 0.1]
total = sum(math.exp(v) for v in z)
print([round(math.exp(v) / total, 3) for v in z])

The outputs add up to 1, which is a quick sanity check worth running on any formula that's meant to give probabilities.

import math
z = [0.0, 0.0]
print(math.exp(z[0]) / sum(math.exp(v) for v in z))

p = [0.9, 0.8, 0.5]

p_all = 0    # the product of every p[i]

print(p_all)

The notation in this phase

Each lesson in this phase introduces its own symbols with code next to them: bold letters like v for vectors, ‖v‖ for a vector's length, u · v for a dot product, ∂ for a partial derivative. When you meet a new one, use this lesson's method: name it, find the loop, write it in Python, and try it on tiny numbers. Keep a short phrasebook of your own as you go: symbol, how to say it, and the line of Python it stands for.

Authors use notation because it's short, not because the idea is hard. Most formulas in this course turn into five lines of Python or fewer, and running those lines on tiny numbers is the fastest way to understand them.