A function is a rule that turns each input into exactly one output, like a machine. It's the most important idea in this whole section: a model is a function from inputs to predictions, and its loss is a function from weights to a single number. This lesson reads f(x) notation, turns a function into a table and a graph, and uses Python's lists and loops to search a table for the best value. You'll also draw your first graph in Python.
What you'll be able to do
Read f(x) notation as a named rule, and work out a function's value for an input
Build a table of values, see it as a graph, and state a function's domain and range
Use lists, for loops, range, len and append in Python to tabulate a function, search for its smallest value and draw its graph
Treat a model's loss as a function of its weight, and find the best of ten weights
Quick check
Have a guess before you start. A machine doubles its input and then adds 1. What comes out when 5 goes in?
10
7
11
12
Show the answer
11
Double 5 to get 10, then add 1: 11. The order of the steps is part of the rule.
The math: a machine with a name
Purpose. Describe exactly how one quantity depends on another.
A function is a rule that gives exactly one output for each input. Picture a machine: a number goes in, the rule is applied, one number comes out.
Functions get names, usually f, g or p, and the input goes in brackets:
p(x)=5x−3p(1)=5×1−3=2
In plain words: p is the rule "times 5, take 3". p(1), read "p of 1", is what comes out when 1 goes in.
p(x) does not mean p times x. The brackets hold the input. It's exactly Python's def p(x): return 5 * x - 3, word for word.
Quick check
From lesson 14: if def f(x): return 3 * x, what is f(2)?
f2
5
6
32
Show the answer
6
f(2) puts 2 in for x: 3 × 2 = 6. Math's f(x) and Python's def f(x) are the same idea.
Watch the machine p(x) = 5x − 3 at work on x = 4. The picture draws the rule as two boxes in a row. The number goes in on the left, and each Step carries it through one box. The number above each gap is the value at that point, and the dot underneath shows how far it has traveled. The cells below do the same arithmetic.
Try it now
Then set the input to 1, or to -2, and press Use these numbers: one input, one output, every time.
From table to graph. A table of values lists inputs and their outputs. For f(x) = x²:
x
−2
−1
0
1
2
f(x)
4
1
0
1
4
Plot each pair as a dot, (x, f(x)), and join them: the table becomes a curve. Here it's a U, lowest at 0.
Domain and range. The domain is the inputs a function accepts; the range is the outputs it can give. x² takes any number, but its outputs never go below 0. The square root takes no negatives (no real number squared gives −4), and 1/x takes anything except 0. Drag the point along x² and watch the output.
Quick check
f(x) = x² − 1. What is f(3)?
35
4
5
8
Show the answer
8
f(3) = 3² − 1 = 9 − 1 = 8.
Quick check
Which number is outside the domain of √x, the square root?
4
−4
0.25
0
Show the answer
−4
No real number squared gives −4, so √(−4) has no real answer. 0, 4 and 0.25 all have square roots: 0, 2 and 0.5.
Shifting and stretching. In f(x) = w·x + b, the two letters move the graph. Changing b slides the whole line up or down without turning it. Changing w tilts it: a bigger w makes it climb faster. These are exactly a model's weight and bias from lesson 14, so training a model means sliding and tilting a graph until it fits the data.
In your own words
In your own words: what makes a rule a function, and what do the domain and range describe?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: tables with lists and loops
Purpose. Work out a function for many inputs at once, then search the results.
A list holds several values in order, in square brackets. len counts them, [i] reads one by its position, counting from 0, and [-1] reads the last. max gives the biggest:
It runs right here in your browser, with nothing to install.
A for loop runs the same lines once for each value. range(-2, 3) gives −2, −1, 0, 1 and 2: it starts at the first number and stops just before the second. Together they print a table of values:
for x inrange(-2, 3):
print(x, x ** 2)
A loop that fills a list. A for loop can also walk through a list: for x in xs: runs once for each value in xs, in order. And a loop can build a list as it goes. Start with an empty list, ys = [], and each time round, ys.append(value) adds one value to its end. Below, ys fills up with the outputs of f(x) = x² − 4x, one for each input in xs. Press Next to run one line at a time, and watch ys grow by one box on each pass.
Searching a table by hand. To find where f is smallest, keep the best so far and update it whenever something beats it. This hand-written search is a pattern you'll use again and again:
deff(x):
return x ** 2 - 4 * x
best_x = 0
best = f(0)
for x inrange(0, 6):
if f(x) < best:
best = f(x)
best_x = x
print(best_x, best)
It starts with x = 0 as the best so far, then tries each x and keeps any that's lower. The answer: the smallest value in the table is −4, at x = 2.
Worked example
Worked example: a table, then its peak
Tabulate g(x) = 10 − (x − 3)² for x from 0 to 6, and find where it's largest.
defg(x):
return10 - (x - 3) ** 2
best_x = 0for x inrange(0, 7):
print(x, g(x))
if g(x) > g(best_x):
best_x = x
print("largest at", best_x, "where g is", g(best_x))
In the lesson you can change this and run it.
The loop prints every row of the table, 1, 6, 9, 10, 9, 6, 1, and checks each against the best so far. The values rise to 10 at x = 3, then fall again: the graph is an upside-down U, peaking where (x − 3)² is 0. Searching for the largest just flips the comparison from < to >.
Quick check
What does this print?
total = 0for n inrange(1, 5):
total = total + n
print(total)
15
10
4
5
Show the answer
10
range(1, 5) gives 1, 2, 3 and 4, stopping before 5. Their total is 10.
Quick check
Which line gives the list's first item?
nums[0]
nums(0)
nums[1]
nums.first
Show the answer
nums[0]
List positions count from 0, so the first item is at position 0.
From two lists to a graph. A pair of lists, inputs and outputs, is exactly what a graph is drawn from, and exactly what a model is trained on: inputs in one list, the right answers in another. Python draws graphs with Matplotlib. import matplotlib.pyplot as plt loads its drawing tools under the short name plt. Then four lines do the work, here and in every graph after this one:
plt.plot(xs, ys) draws a line through the points (xs[0], ys[0]), (xs[1], ys[1]) and so on, in order.
plt.xlabel and plt.ylabel name the axes.
plt.show() shows the finished graph.
import matplotlib.pyplot as plt
xs = [0, 1, 2, 3, 4, 5]
ys = []
for x in xs:
ys.append(x ** 2 - 4 * x)
plt.plot(xs, ys)
plt.xlabel("x")
plt.ylabel("f(x)")
plt.show()
It's the stepper's table, drawn: a U like x² in the math part, lowest at x = 2.
Quick check
Read the graph. At which inputs is f(x) exactly 0?
x = 0 and x = 4
x = 1 and x = 3
x = 4 and x = 5
Only at x = 2
Show the answer
x = 0 and x = 4
The curve sits on 0 at x = 0 and again at x = 4, matching the 0s in ys. In between it dips below 0, down to −4 at x = 2.
Drill
A table and its lowest point
Fix the loop so it tries x from 0 to 8 (both included) and finds where h(x) = x² − 6x + 10 is smallest.
defh(x):
return x ** 2 - 6 * x + 10
best_x = 0for x inrange(0, 8):
if h(x) > h(best_x):
best_x = x
print(best_x, h(best_x))
In the lesson, your code is checked as you write it.
In your own words
Why does a search start with a "best so far" and update it, instead of looking at all the values at once?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: the loss is a function of the weight
Purpose. A model is a function from inputs to predictions. Its loss is a second function: it takes the model's weight and gives one number, how wrong the model is on the data.
Three days of tent rentals, by temperature: (10, 22), (20, 41) and (30, 59). A model ŷ = w·x predicts rentals; its loss for a weight w is the average squared error over the three days:
L(w)=31[(10w−22)2+(20w−41)2+(30w−59)2]
In plain words: for each day, the miss, squared; then the average of the three.
xs = [10, 20, 30]
ys = [22, 41, 59]
defloss(w):
total = 0for i inrange(len(xs)):
total = total + (w * xs[i] - ys[i]) ** 2return total / len(xs)
for t inrange(15, 25):
w = t / 10print(w, round(loss(w), 2))
Quick check
In the table above, which weight gives the lowest loss?
2.4
2.0
1.5
All are about the same
Show the answer
2.0
The loss falls from 122.0 at w = 1.5 to 2.0 at w = 2.0, then rises again. The bottom of the table is w = 2.0.
The table is a graph. Put each row on a graph, weight across and loss up, and the ten rows make a bowl. Choosing the weight at its bottom is exactly the hand-written search from the Python part. Look at how the dots fall steeply, flatten out near the bottom, and climb again. Then press Next to zoom in on the bottom three.
One row by hand. Before you trust the table, check its best row yourself: at w = 2, work out each day's miss, square it, and predict what this prints.
Quick check
What does this print?
xs = [10, 20, 30]
ys = [22, 41, 59]
total = 0for i inrange(len(xs)):
total = total + (2 * xs[i] - ys[i]) ** 2print(total, total / len(xs))
6 2.0
0 0.0
6 6.0
2 6.0
Show the answer
6 2.0
With w = 2 the misses are −2, −1 and 1, which square to 4, 1 and 1: a total of 6, and an average of 6 ÷ 3 = 2.0.
Drill
The best of ten weights
With the same three days, search the ten weights w = 1.5, 1.6, …, 2.4 and set best_w to the one with the lowest loss and best to its loss.
xs = [10, 20, 30]
ys = [22, 41, 59]
defloss(w):
total = 0for i inrange(len(xs)):
total = total + (w * xs[i] - ys[i]) ** 2return total / len(xs)
best_w = 1.5
best = loss(1.5)
for t inrange(15, 25):
w = t / 10if loss(w) > best:
best = loss(w)
print(best_w, best)
In the lesson, your code is checked as you write it.
In the real tools. Real models have millions of weights, so training can't try every value. Lesson 79 shows how they walk downhill instead. But the idea is the same: the loss is a function, and training looks for its lowest point. In PyTorch, the library most models are built with, the loss is a function you call, on the model's predictions and the right answers. Here is this lesson's loss, written that way:
In your own words
What are the input and the output of a model's loss function?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write