Calculus questions often come backward. You know the slope you want and need the x. You know one point on a curve and need its constant. You know the step a training run took and need its learning rate. Once the calculus is done, each one is an ordinary rearrangement, with two conditions to watch: a slope you divide by can't be 0, and a square gives a ± to choose from.
What you'll be able to do
Find where a slope is 0 by setting f′(x) = 0 and making x the subject, keeping both roots when a square appears
Rearrange the tangent line for the step to a target, and use it as Newton's method for roots and square roots
Turn the chain rule around for an inner slope, and fix an integral's constant or upper limit from a known value
Solve a gradient step for its learning rate, and find the rate that reaches a bowl's bottom in one step
Find a model's best weight in closed form, and see Newton's method behind second-order optimizers
Quick check
Have a guess before you start. A curve's slope is 2x − 10. At which x is the slope 0?
10
−5
5
20
Show the answer
5
2x − 10 = 0, so 2x = 10 and x = 5. Setting a slope to 0 and solving is the first rearrangement in this lesson: it finds a lowest or highest point.
The math: solve the calculus for the letter you need
Purpose. Transposing in calculus: turn each formula of the module around for the letter a question asks about.
A quick recap: f′(x) is the slope of f at x (lesson 73), and f″(x) is how fast that slope changes, the curvature (lesson 75). Once the slope is worked out, the usual moves finish the job.
You know
You want
Rearranged
the slope is 0
x
solve f′(x) = 0 for x
the tangent line
the step h
h = (target − f(x))/f′(x)
dy/dx and dy/du
du/dx
du/dx = (dy/dx)/(dy/du)
one point on F
the constant C
C = F(x₀) − the rest of F at x₀
a gradient step
the learning rate
η = (w − w_new)/g
A lowest point. A curve f(x) = ax² − bx + c has slope f′(x) = 2ax − b. At the bottom the slope is 0, so 2ax − b = 0. Add b to both sides: 2ax = b. Divide by 2a: x = b/(2a). The conditions: a can't be 0, and a must be above 0 for a bowl with a lowest point.
Quick check
From an earlier lesson: what is the slope of 2x² − 40x + 300?
4x − 40
4x² − 40
4x − 40 + 300
2x − 40
Show the answer
4x − 40
The power comes down and drops by one: 2x² gives 4x. −40x gives −40, and the constant 300 gives 0.
The tent-rental shop sews its own tents. When it makes x tents a day, the cost per tent is f(x) = 2x² − 40x + 300 dollars: it falls as the workshop gets busy, then climbs with overtime. Step through the slope set to 0.
Try it now
The widget writes the curve as ax² + bx + c, so its b is −40. Set b to −60 and press Use these numbers: x moves to 60 ÷ 4 = 15. Set a to −2: f″ turns negative, and the flat point becomes a highest point.
Symbols and formula.
f(x)=ax2−bx+c⇒f′(x)=2ax−b=0⇒x=2ab
In plain words: find the slope, set it to 0, then make x the subject with the usual moves.
When a square appears. The slope of x³ is 3x². Where is that slope 12? Divide by 3: x² = 4. Take the square root: x = ±2. Both work, since (−2)² is 4 too. A square hides the sign, so undoing it gives two answers. The question then decides which ones to keep.
Quick check
f(x) = x³ − 12x. Where is its slope 0?
x = 2 or x = −2
x = 2 only
x = 12
x = 4 or x = −4
Show the answer
x = 2 or x = −2
f′(x) = 3x² − 12 = 0, so x² = 4 and x = ±2. Check: 3 × (−2)² − 12 = 0. This curve has a top at −2 and a bottom at 2.
An area that must hit a target. A runner's speed is v = 2t meters per second, t seconds after the start. The distance covered by time a is the area under the speed line from 0 to a (lesson 84). That area is a triangle with base a and height 2a, so ∫₀ᵃ 2t dt = ½ × a × 2a = a². When is the distance 9 meters? a² = 9 gives a = ±3. Both solve the equation, but −3 seconds is before the start, so keep a = 3.
The machine below shows why a square needs a ±. It builds 3x² from x = −2, then peels it back.
A step to a target. Near x, the tangent line stands in for the curve (lesson 76): f(x + h) ≈ f(x) + f′(x)h. To land on a target, set that equal to the target and make h the subject. Take f(x) away, then divide by f′(x): h = (target − f(x))/f′(x).
Newton's method. To find where f(x) = 0, use a target of 0. Then h = −f(x)/f′(x), and Newton's step is
xnew=x−f′(x)f(x)
The tangent is only a stand-in, so one step lands close, not exactly. Repeat it.
A square root. √10 is the x above 0 with f(x) = x² − 10 = 0, and f′(x) = 2x. From x = 3: 3 − (9 − 10)/6 = 3.166667. Once more gives 3.162281, and √10 = 3.162278.
The condition. f′(x) can't be 0. At x = 0 the tangent of x² − 10 is flat: it never reaches 0, and the step divides by 0.
Quick check
Newton's step for x² − 10 = 0 starts at x = 0. What goes wrong?
x² − 10 has no root
It divides by f′(0) = 0
It lands exactly on √10
It finds −√10 instead
Show the answer
It divides by f′(0) = 0
f′(x) = 2x is 0 at x = 0. The tangent there is flat, so it never meets 0, and x − f(x)/f′(x) divides by 0. Start anywhere else.
The chain rule, turned around. dy/dx = (dy/du)(du/dx) multiplies two slopes (lesson 74): y depends on an inner u, and u on x. If you know the whole slope and the outer one, divide: du/dx = (dy/dx)/(dy/du), as long as dy/du isn't 0. For y = u² at u = 3, dy/du = 2u = 6. If dy/dx = 12, then du/dx = 12 ÷ 6 = 2.
A constant from a point. An antiderivative of f is a function F whose slope is f: it undoes taking the slope, the way summing undoes differencing in lesson 84. The runner's distance t² is one for the speed 2t. But x² + 4 and x² − 7 also have slope 2x, since a constant adds no slope. So every antiderivative of 2x is F(x) = x² + C, where C, the constant of integration, is unknown. One known point fixes it. If F(1) = 5, then 1² + C = 5, so C = 4: the curve x² lifted 4 units.
Named mistakes. Dropping the − root of x² = 4. Writing Newton's step with a plus. Turning the chain rule upside down, (dy/du)/(dy/dx). Taking C to be 5, the value of F, without taking away the x² part.
In your own words
Solving 3x² = 12 gives two answers, but Newton's method for √10 from x = 3 finds only one. Why the difference?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: rearranged formulas as functions
Purpose. Write each turned-around formula as a function. Put its condition first, and check it with a round trip.
defnewton_sqrt(n, x, steps):
for i inrange(steps):
x = x - (x * x - n) / (2 * x)
print(i + 1, round(x, 6))
return x
r = newton_sqrt(10, 3, 3)
print(round(r * r, 9))
It runs right here in your browser, with nothing to install.
Each pass is one Newton step for f(x) = x² − n, with f′(x) = 2x.
The error shrinks fast: about 0.004 after one step, 0.000003 after two.
Squaring the answer gives back 10. That's the round trip.
Worked example
Worked example: a constant and a time from integrals
Two rearrangements, each with its condition: C = F(x₀) − x₀² for F(x) = x² + C, and the time T with T² = distance, keeping T at or above 0.
import math
defconstant_from_point(x0, F0):
return F0 - x0 ** 2deftime_to_cover(distance):
if distance < 0:
returnNonereturn math.sqrt(distance)
C = constant_from_point(1, 5)
print(C, 1 ** 2 + C)
print(time_to_cover(9), time_to_cover(0), time_to_cover(-4))
In the lesson you can change this and run it.
C = 5 − 1 = 4, and putting x = 1 back into x² + 4 gives 5: the point is on the curve. math.sqrt returns only the root at or above 0, which is the one a time needs, so 9 meters takes 3.0 seconds. No square is negative, so a distance of −4 has no time at all, and the function says None instead of crashing.
Quick check
What does this print?
x = 3print(round(x - (x * x - 10) / (2 * x), 4), round(x + (x * x - 10) / (2 * x), 4))
2.8333 3.1667
3.1667 3.1667
3.0 3.0
3.1667 2.8333
Show the answer
3.1667 2.8333
f(3) = −1 and f′(3) = 6, so f/f′ = −0.1667. Subtracting it moves up to 3.1667, toward √10 ≈ 3.1623. Adding it moves down to 2.8333, away from the root.
Quick check
dy/dx = 12 and dy/du = 6 are stored as dy_dx and dy_du. Which line gives du/dx?
dy_dx * dy_du
dy_du / dy_dx
dy_dx / dy_du
dy_dx - dy_du
Show the answer
dy_dx / dy_du
dy/dx = (dy/du)(du/dx), so divide both sides by dy/du: 12 ÷ 6 = 2. Check: 6 × 2 = 12.
Drill
Fix the Newton step and the inner slope
newton_step adds the correction, and inner_slope divides upside down. Fix both, and return None when the number you divide by is 0.
In the lesson, your code is checked as you write it.
In your own words
Each function here checks the number it divides by before dividing. What would go wrong without that check?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: the best weight and the learning rate
Purpose. Training runs the calculus forward. Planning a run, or reading its logs, runs it backward.
The learning rate from a step. A gradient step is w_new = w − ηg, with g the slope of the loss (lesson 79). Make η the subject: take w away and change signs, so ηg = w − w_new. Divide by g: η = (w − w_new)/g, as long as g isn't 0. Read backward, a logged step tells you the rate. Read forward: a step's size is η × |g|, so a step of a size s you choose needs η = s/|g|.
Quick check
A log shows a weight moving from 1.0 to 0.94 with gradient 1.5. What does this print?
w, w_new, g = 1.0, 0.94, 1.5
eta = (w - w_new) / g
print(round(eta, 4), round(w - eta * g, 4))
-0.04 1.06
0.04 0.94
0.04 1.06
0.09 0.94
Show the answer
0.04 0.94
w − w_new = 0.06, and 0.06 ÷ 1.5 = 0.04. Putting it back, 1.0 − 0.04 × 1.5 = 0.94: the round trip lands on the logged weight.
The best weight in one line. A model ŷ = w·x predicts tent rentals from temperature, scored by L(w) = Σ(wx − y)². Its slope is L′(w) = 2Σx(wx − y) = 2(wΣx² − Σxy). Set it to 0 and make w the subject:
w=∑x2∑xy
In plain words: the best weight is the inputs times the targets, added up, over the inputs squared, added up.
This is a closed-form answer: no training loop at all. Its condition: Σx² isn't 0, so not every x is 0. Here it is on the three days of tent data from lesson 72. That lesson averaged the squared errors; adding them instead doesn't move the bottom, which it found between 2 and 3:
import numpy as np
x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 7.0])
defslope(w):
return2 * np.sum(x * (w * x - y))
best = np.sum(x * y) / np.sum(x * x)
rate = 1 / (2 * np.sum(x * x))
print(round(best, 4), slope(0.0))
print(round(0.0 - rate * slope(0.0), 4))
Quick check
In w = Σxy/Σx², when is there no best weight?
When every x is 0
When w comes out below 0
When there are 3 points
When every y is 0
Show the answer
When every x is 0
Σx² is 0 only when every x is 0. Then the loss doesn't change with w at all, so no weight is better than another.
One step to the bottom. That second print used a special rate. A loss L = a(w − t)² is a bowl with its bottom at t, and slope 2a(w − t). Ask for the η whose step lands on t: t = w − η × 2a(w − t). Then w − t = 2aη(w − t). Divide by w − t (when w isn't already t): η = 1/(2a). Our tent loss is this bowl plus a constant, with a = Σx² = 14, so η = 1/28, and one step from 0 lands on 2.2143. The widget calls the weight x.
Newton's idea in optimizers. A lowest point is where the slope L′ is 0. So use Newton's step on the slope itself: w_new = w − L′(w)/L″(w). That's gradient descent with η = 1/L″. For a bowl, L″ = 2a: Newton reaches a bowl's bottom in one step. Optimizers that use L″ are called second-order methods. With a million weights, L″ is the Hessian (lesson 83), a table of second slopes with a million rows and a million columns: far too big to store. Quasi-Newton methods such as L-BFGS build a rough stand-in from the last few steps and gradients. They suit smaller models; large networks mostly train with gradient descent and its variants. The condition is Newton's again: L″ can't be 0, and it must be above 0 to head for a bottom.
Drill
The best weight and the one-step rate
Write best_weight(x, y), the w that makes Σ(wx − y)² smallest (return None when Σx² is 0), and one_step_rate(x), the rate 1/(2Σx²) that reaches it in one step.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
Learn to make any letter of any formula the subject: layers undone in reverse order, fractions, a letter that appears twice, powers, roots, quadratics, logs, inequalities, trigonometry, vectors, matrices, calculus, probability, statistics and the formulas inside AI models, each checked in Python.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write