The derivative asked how fast something changes. The integral asks the reverse: given a rate, what's the total? A car's speed over an hour adds up to the distance it covered; a curve's heights, added over a stretch, give the area underneath. The method is the same idea as everything else in this module: cut the problem into thin slices, each easy, and add them up, with thinner slices giving a better answer. In ML, area turns up in two important places: the probability of a continuous value landing in a range is an area under a curve, and one of the most common scores for a classifier, the AUC, is literally an area under a curve.
What you'll be able to do
Estimate the area under a curve by cutting it into thin strips and adding them up
Use trapezoids for a better estimate, and watch the answer settle as the strips get thinner
Write a trapezoid sum in Python, plot the trapezoids' tops against the curve, and see that adding up differences undoes taking them
Find a probability as an area under a bell curve, and an AUC score as an area under a ROC curve, in NumPy and with torch.trapezoid
Quick check
Have a guess before you start. A cyclist rides at 12 km/h for half an hour, then 20 km/h for half an hour. How far did they go?
20 km
16 km
8 km
32 km
Show the answer
16 km
12 × 0.5 + 20 × 0.5 = 6 + 10 = 16 km: each speed times its time, added up. That's an integral in miniature.
The math: strips, added up
Purpose. A total from a rate, an area from a curve.
To find the area under f between x = a and x = b, cut the stretch into n strips of width Δx = (b − a)/n. Each strip is close to a rectangle, height f(x) and width Δx. Add them up:
area≈i∑f(xi)Δx
In plain words: height times width for each thin strip, all added. It's lesson 26's Σ again.
For x² from 0 to 2 with 4 strips, using each strip's left edge for the height: 0.5 × (0 + 0.25 + 1 + 2.25) = 1.75. The true area is 8/3 ≈ 2.667, so left edges underestimate a rising curve.
Quick check
From lesson 26: what is Σ i for i = 1 to 4?
10
4
24
16
Show the answer
10
1 + 2 + 3 + 4 = 10. An integral's strip sum is the same kind of Σ, adding heights times widths.
Trapezoids. Using the average of each strip's two edges as its height turns each rectangle into a trapezoid, which follows the curve much better. Step through x² from 0 to 2 with 4 trapezoids.
The picture. The drawing starts with the 4 strips as plain rectangles, each as tall as its left edge: the 1.75 from above. Each step lays a trapezoid over one strip, its slanted top joining the curve at both edges, and writes its area. Watch how much closer a slanted top follows the curve than a flat one. The PyTorch tab adds every strip at once.
Try it now
Then set the strips to 8: the total moves from 2.75 toward the true 2.667, and with 12 it moves closer still.
Symbols and formula. As the strips get thinner, the sum settles on one number, the integral:
∫abf(x)dx=n→∞limi∑f(xi)Δx
In plain words: the ∫ is a stretched S for "sum", and dx is a strip's width shrunk toward zero.
For x² from 0 to 2: 4 strips give 2.75, 8 give 2.6875, and the limit is 8/3. Summing undoes differencing: if you take a list's differences (lesson 70) and add them back up, you recover how far the list has come from its start. That's the deep link between the two halves of calculus: a rate, summed, gives back the total change.
Quick check
A speed is 0, 2, 4, 4 and 3 km/h, read every half hour. Using trapezoids, about how far was traveled?
6.5 km
4 km
5.75 km
13 km
Show the answer
5.75 km
0.5 × (0/2 + 2 + 4 + 4 + 3/2) = 0.5 × 11.5 = 5.75. Each end counts half, since it borders only one strip.
In your own words
In your own words: why do thinner strips give a better estimate of the area?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: a trapezoid sum
Purpose. Add up strips in a line or two, for any curve you can evaluate.
Every interior point is the edge of two strips and counts fully; the two ends count half:
import numpy as np
deftrapezoid(y, dx):
return dx * (np.sum(y) - (y[0] + y[-1]) / 2)
for n in [2, 4, 8]:
xs = np.linspace(0, 2, n + 1)
print(n, trapezoid(xs ** 2, 2 / n))
It runs right here in your browser, with nothing to install.
y[-1] is the last value, counting from the end. The estimates, 3, 2.75 and 2.6875, close in on 8/3. NumPy has a built-in version, np.trapezoid (formerly np.trapz), that does the same.
Code flow.trapezoid adds every strip in one line. Here is the same sum one strip at a time, as the math part's flow did it. Step through it and watch strip take each trapezoid's area, and total grow by it: the flow's running totals.
Summing undoes differencing.np.cumsum gives a running total, so it reverses np.diff:
import numpy as np
v = np.array([2, 5, 9, 10])
d = np.diff(v)
print(d.tolist(), np.cumsum(d).tolist(), (v[0] + np.cumsum(d)).tolist())
Add the start back on and the original values return (all but the first, which the differences never knew). A named mistake: forgetting the width. Adding heights without multiplying by Δx gives a number that grows with the number of strips, instead of settling. If doubling n roughly doubles your "area", the width is missing.
Worked example
Worked example: three areas to two decimals
Estimate three areas by trapezoids, raising n until the answer stops changing in the second decimal.
import math
import numpy as np
deftrapezoid(y, dx):
return dx * (np.sum(y) - (y[0] + y[-1]) / 2)
defarea(f, a, b, n):
xs = np.linspace(a, b, n + 1)
return trapezoid(f(xs), (b - a) / n)
for n in [4, 16, 64]:
print(n, round(area(np.exp, 0, 1, n), 4), round(area(np.sqrt, 0, 4, n), 4), round(area(np.sin, 0, math.pi, n), 4))
In the lesson you can change this and run it.
eˣ from 0 to 1 settles near 1.7183 (e − 1); √x from 0 to 4 near 5.33 (16/3), slowest because √x is steep at 0; sin x from 0 to π near 2. Each column moves less as n grows, which is how you know the strips are thin enough: the answer has stopped changing at the precision you need.
Quick check
What does this print?
import numpy as np
y = np.array([1.0, 3.0, 5.0])
print(0.5 * (np.sum(y) - (y[0] + y[-1]) / 2))
3.0
6.0
2.5
4.5
Show the answer
3.0
0.5 × (9 − 3) = 3.0: two trapezoids of width 0.5 under a straight line, which is exact.
Why not just use rectangles? Left-edge rectangles miss a rising curve on every strip, always on the same side, so their errors pile up. Trapezoids follow the curve's slope across each strip, so most of that error disappears: for x² from 0 to 2 with 4 strips, rectangles are off by 0.92 and trapezoids by only 0.08. Better methods exist, but trapezoids are the everyday choice because they're simple and good.
Plot it. Draw the curve and the trapezoids' tops together. The second plt.plot joins the 5 edge points with straight lines, and "o-" marks each point with a dot:
Read the plot. Between two dots, where does a trapezoid's straight top sit compared with the curve?
A little above it, so the total comes out too big
Far above it, by about 1 at every point
Exactly on it, so the total is exact
A little below it, so the total comes out too small
Show the answer
A little above it, so the total comes out too big
x² bends upward, so the straight line joining two of its points runs just above it. Every trapezoid is a little too tall, which is why 4 of them give 2.75 instead of 2.667.
Drill
A trapezoid area
Fix trapezoid so it counts the ends half and multiplies by the width. Then set a1 to the area under 3x² from 0 to 1 with 4 strips, rounded to 4 places.
In the lesson, your code is checked as you write it.
In your own words
Why do the two end values count half in a trapezoid sum?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: probability and AUC are areas
Purpose. Two of ML's everyday numbers are areas under curves.
Probability as area. For a value that can be any number, like a height or a measurement error, chances come from a curve, a density: the probability of landing between a and b is the area under it there. For the standard bell curve (lesson 93), the share between −1 and 1 is about 0.683.
P(a≤X≤b)=∫abp(x)dx
In plain words: probability is area under the density curve.
import math
import numpy as np
defbell(x):
return np.exp(-x ** 2 / 2) / math.sqrt(2 * math.pi)
deftrapezoid(y, dx):
return dx * (np.sum(y) - (y[0] + y[-1]) / 2)
for n in [4, 20]:
xs = np.linspace(-1, 1, n + 1)
print(n, round(trapezoid(bell(xs), 2 / n), 4))
With 20 strips the estimate is 0.6823, close to the true 0.6827.
Quick check
A classifier's ROC curve passes through (0, 0), (0.1, 0.6), (0.3, 0.9) and (1, 1). What is its area by trapezoids?
AUC. A classifier's ROC curve plots the share of true cases it catches against the share of false alarms, as its threshold slides. The area under that curve, the AUC, is one number for how well it separates the classes: 1.0 is perfect, 0.5 is guessing. Its points come at uneven spacing, so each strip has its own width: np.diff of the x-values gives them.
Try it on a model that scores 200 customers for whether they'll churn, that is, leave. The table counts what it gets right and wrong at your threshold, and the ROC curve beside it is every threshold at once, with its AUC.
Reading an AUC. An AUC of 0.9 means that if you pick one positive and one negative example at random, the classifier ranks the positive one higher 90% of the time. That's why it's popular: it doesn't depend on any one threshold, only on how well the scores separate the classes, which is exactly what a ranking asks for.
Quick check
A ROC curve passes through (0, 0), (0.25, 0.8) and (1, 1). What does this print?
Both match the NumPy cells: 0.775 for the ROC curve in the check above, and 0.6823 for the bell curve with 20 strips. torch.linspace(-1, 1, 21) gives the same 21 edges as np.linspace, and dtype=torch.float64 keeps NumPy's precision. Handing over the x-values, never one width, is what lets the same call do both.
Drill
The area under a ROC curve
Set widths to the strip widths of the false-alarm rates, heights to the average catch rate of each strip, and auc to their sum of products, rounded to 4 places. tpr[1:] drops the first value and tpr[:-1] the last.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write