Big rules are built from small ones: do one function, then feed its answer into the next. That's called composing, and a neural network is exactly that, a chain of simple functions, layer after layer. Running a chain backward is the inverse, and it's how data scaled for a model gets turned back into numbers people understand. You'll also see why some functions, like squaring and ReLU, can't be undone at all without a restriction. Every step is checked in Python.
What you'll be able to do
Compose functions, g(f(x)), and see why the order matters
Find an inverse function by undoing each step in reverse, and check it
Chain functions in Python, and check an inverse on a list of inputs with a loop
See a neural network as a chain of functions, and undo a scaling step
Quick check
Have a guess before you start. f doubles a number and g adds 3. What is g(f(5))?
8
16
10
13
Show the answer
13
Work from the inside out: f(5) = 10, then g(10) = 13. The function nearest the number goes first.
The math: chains of machines
Purpose. Build big rules from small ones, and run rules backward.
Composing two functions means feeding the output of one into the other. g(f(x)) is read from the inside out: do f first, then g. Its other name is (g ∘ f)(x), "g after f".
Order matters. Doubling then adding 3 isn't the same as adding 3 then doubling: g(f(5)) = 13 but f(g(5)) = 16. Putting on socks then shoes is different from shoes then socks.
An inverse runs a function backward: it takes the output and gives back the input. The inverse of f is written f⁻¹. That little −1 doesn't mean "one over"; it means "undo".
Quick check
From an earlier lesson: make x the subject of y = 2x + 1.
x = 2y + 1
x = y / 2 − 1
x = (y − 1) / 2
x = (y + 1) / 2
Show the answer
x = (y − 1) / 2
Take 1 away, then divide by 2. That rearrangement is exactly the inverse of f(x) = 2x + 1.
Finding an inverse. List the steps the function does, then undo them in reverse order. f(x) = 2x + 1 does "times 2, then add 1", so f⁻¹ does "take 1 away, then divide by 2": f⁻¹(y) = (y − 1)/2. Watch a three-step chain run forward, then undo itself step by step.
In the picture, each box is one step. The number is written in the gap it has reached, with a dot under it. When the chain has run, it turns round: every box shows its opposite, the arrows point back, and the number runs right to left until it reaches the start.
Try it now
Then set the input to 5 and press Use these numbers: the chain still brings it back. Swap two steps of a chain by hand and the answer changes.
Symbols and formula. Composing, and what an inverse promises:
(g∘f)(x)=g(f(x))f−1(f(x))=x
In plain words: g after f means do f, then g. Running a function and then its inverse gets you back where you started.
Some functions have no inverse. x² sends both 3 and −3 to 9, so seeing 9, you can't tell which went in. The fix is to restrict the inputs: for x ≥ 0 only, the square root undoes squaring. That's why the chain above worked: 2 × 3 + 1 was positive.
Quick check
f(x) = x + 5 and g(x) = 3x. Which is bigger: g(f(1)) or f(g(1))?
f(g(1)): 18 against 8
g(f(1)): 18 against 8
They're equal: both are 18
They're equal: both are 8
Show the answer
g(f(1)): 18 against 8
g(f(1)): add 5 to get 6, then triple to 18. f(g(1)): triple to 3, then add 5 to get 8. Order matters.
Quick check
What is the inverse of h(x) = x/4 − 2?
h⁻¹(y) = 4 / (y + 2)
h⁻¹(y) = y × 4 + 2
h⁻¹(y) = (y + 2) × 4
h⁻¹(y) = (y − 2) / 4
Show the answer
h⁻¹(y) = (y + 2) × 4
h divides by 4, then takes 2 away. Undo in reverse: add 2 back, then multiply by 4. Check: h(7) = −0.25, and (−0.25 + 2) × 4 = 7.
In your own words
In your own words: why can't x² be undone without a restriction?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: functions calling functions
Purpose. A chain in code is a call inside a call, and a loop can check an inverse on many inputs at once.
deff(x):
return2 * x
defg(x):
return x + 3print(g(f(5)), f(g(5)))
It runs right here in your browser, with nothing to install.
Python works the inner call first, just like the math: f(5) is 10, then g(10) is 13. Step through it below. The first two lines do the chain one call at a time; the third does it as a call inside a call. Press Next and watch which function Python steps into first on that line.
An inverse is a function too. Writing both lets one check the other:
Both orders give back 4: the inverse undoes h, and h undoes the inverse.
Checking an inverse on a list. One input could match by luck, so check several. A loop over a list does it in three lines:
defh(x):
return2 * x + 1defh_inv(y):
return (y - 1) / 2for x in [0, 3, -2, 7.5]:
print(x, h_inv(h(x)) == x)
A named mistake: undoing in the same order. For h(x) = 2x + 1, dividing by 2 first and then taking 1 away gives (y/2) − 1, which isn't the inverse: h_inv would return 3.5 for y = 9, not 4. The loop's True or False catches it.
Worked example
Worked example: a three-step pipeline and its way back
Build the chain times 2, take 1, square as three small functions, run 3 through them, then undo it.
defstep1(x):
return x * 2defstep2(x):
return x - 1defstep3(x):
return x ** 2defback(y):
return (y ** 0.5 + 1) / 2
out = step3(step2(step1(3)))
print(out, back(out))
In the lesson you can change this and run it.
step3(step2(step1(3))) reads inside out: 3 becomes 6, then 5, then 25. back undoes the three steps in reverse: square root (5.0), add 1 (6.0), divide by 2 (3.0). Writing each step as its own small function is how real data pipelines are built, so each piece can be tested and swapped on its own.
Quick check
What does this print?
deff(x):
return x - 4defg(x):
return x * 10print(f(g(2)), g(f(2)))
-20 16
-20 -20
16 16
16 -20
Show the answer
16 -20
f(g(2)): times 10 is 20, minus 4 is 16. g(f(2)): minus 4 is −2, times 10 is −20.
Quick check
Which line checks that f_inv undoes f at x = 6?
print(f(f_inv(6)) == f(6))
print(f(6) == f_inv(6))
print(f_inv(6) == 6)
print(f_inv(f(6)) == 6)
Show the answer
print(f_inv(f(6)) == 6)
Run f, then its inverse, and compare with where you started.
Why check with a tolerance? For h(x) = 2x + 1 the round trip is exact, because doubling and halving lose nothing. With other numbers, floats are stored a hair off, and h_inv(h(x)) == x can be False even when the inverse is right. So careful checks compare with a small tolerance, abs(a - b) < 1e-9, as lesson 18 did. A right inverse passes, and a wrong one fails by far more than a hair.
Drill
Fix an inverse, then check it
k(x) = 5x − 2. Fix k_inv so it undoes k, and set all_back to whether the round trip works for every x in tests.
defk(x):
return5 * x - 2defk_inv(y):
return y / 5 + 2
tests = [0, 1, -3, 4.4]
all_back = Truefor x in tests:
if k_inv(k(x)) != x:
all_back = Trueprint(all_back)
In the lesson, your code is checked as you write it.
In your own words
Why is it useful to write each step of a chain as its own small function?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: networks are chains
Purpose. A neural network is a chain of functions, and data preparation is undone at the end.
Scaling and unscaling. Models learn best from numbers near 0. So inputs are often scaled: take away a typical value (the mean) and divide by a typical spread. Predictions come out scaled too, and the inverse turns them back:
s=spreadx−meanx=s×spread+mean
In plain words: scaling takes the mean away then divides; unscaling multiplies back then adds the mean.
Temperatures are scaled with mean 20 and spread 5. A model predicts a scaled value of −1. What temperature is that?
15
−5
25
19
Show the answer
15
Unscale: −1 × 5 + 20 = 15. One spread below the mean.
A tiny network, one number at a time. A neuron is a chain of two functions: the line w·x + b, then ReLU from lesson 18. A network chains layers of them. Here's a two-layer chain on one number:
defrelu(x):
if x <= 0:
return0return x
deflayer1(x):
return relu(2 * x - 1)
deflayer2(h):
return3 * h + 0.5print(layer2(layer1(2)), layer2(layer1(0)))
For x = 2, layer 1 gives relu(3) = 3 and layer 2 gives 9.5. For x = 0, layer 1 gives relu(−1) = 0, so layer 2 gives only its bias, 0.5.
The whole chain, from degrees to rentals. Put scaling in front and unscaling behind, and the tiny network turns a temperature into rentals. Inputs are scaled with the temperatures' mean 20 and spread 5; outputs are unscaled with mean 40 and spread 2. These numbers, like the network's four, are picked by hand to keep the arithmetic easy: a trained network learns its numbers from the shop's data, and real rentals spread far wider than 2. Each row prints every value along the way:
defrelu(z):
if z <= 0:
return0return z
for temp in [30, 20, 10]:
s = (temp - 20) / 5
z = 2 * s - 1
h = relu(z)
out = 3 * h + 0.5print(temp, s, z, h, out, out * 2 + 40)
30 degrees comes out as 59.0 rentals, and 20 degrees as 41.0. The scaled inputs, 2.0 and 0.0, are the x = 2 and x = 0 from above. But 10 degrees comes out as 41.0 too: ReLU turned −1.0 and −5.0 into the same 0. Seeing 41.0, you can't tell which day went in, so this chain has no inverse. The picture runs the chain on two of these days.
In the real tools. In PyTorch, the library most models are built with, a chain of parts is nn.Sequential: each part's output is the next part's input, g(f(x)) for a whole network. Here is the tiny network with its four numbers set to 2, −1, 3 and 0.5, run on the three scaled days:
Order matters inside a network, too. A neuron does its line first, then ReLU: relu(w·x + b). Swapping the order, w·relu(x) + b, makes a different function: it cuts negative inputs to 0 before they're weighed, instead of cutting negative totals. Real networks always weigh and add first, then apply the activation, because the activation is meant to judge the total. Predict both orders below.
Quick check
What does this print?
defrelu(x):
if x <= 0:
return0return x
print(relu(3 * -1 + 4), 3 * relu(-1) + 4)
4 4
4 1
1 1
1 4
Show the answer
1 4
Line then ReLU: 3 × (−1) + 4 = 1, and ReLU keeps 1. ReLU then line: ReLU(−1) = 0, then 3 × 0 + 4 = 4. Different orders, different answers.
Drill
Scale, run, unscale
Write neuron(x, w, b) as relu(w·x + b). Then set out to the neuron with w = 3 and b = −4 run on 2, and temp to the unscaled value of 1.6 with mean 20 and spread 5.
defrelu(x):
if x <= 0:
return0return x
defneuron(x, w, b):
return w * relu(x) + b
defunscale(s, mean, spread):
return (s + mean) * spread
out = neuron(2, 3, -4)
temp = unscale(1.6, 20, 5)
print(out, temp)
In the lesson, your code is checked as you write it.
In your own words
Why is a neural network called a chain of functions?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write