An equation pins a number down; an inequality says where a number may sit. "At least 18", "below freezing" and "between 0 and 1" are inequalities. They're how rules and limits are written, and in code they become decisions: if this, do that. Machine learning is full of them: a score of at least 0.5 means yes, a step too big gets clipped, and the most common activation, ReLU, is a decision on every number.
What you'll be able to do
Read and solve inequalities, flipping the sign when multiplying or dividing by a negative
Describe a range of numbers, with each end in or out, and read |x| < k as a range
Make decisions in Python with if, elif and else, and combine conditions with and, or and not
Write a threshold, a clip and ReLU, the decisions inside a model
Quick check
Have a guess before you start. Which numbers make −2x < 6 true?
x < −3
x < 3
x > 3
x > −3
Show the answer
x > −3
Dividing both sides by −2 flips the sign: x > −3. Check with x = 0: −2 × 0 = 0, and 0 < 6 is true. 0 is above −3.
The math: where a number may sit
Purpose. State a rule about a range of numbers, not one number.
The four signs: < "less than", ≤ "less than or equal to", > "greater than", ≥ "greater than or equal to". x ≥ 18 includes 18; x > 18 doesn't. On a number line, a filled dot marks an end that's included, and an open dot one that isn't.
A range, or interval, has two ends: 0 ≤ x < 10 means x is at least 0 and below 10. It includes 0 but not 10.
Solving an inequality works like solving an equation, with one change. Adding or taking away on both sides is fine, and so is multiplying or dividing by a positive number. But multiplying or dividing by a negative flips the sign: −2x < 6 becomes x > −3.
Quick check
From an earlier lesson: which is bigger, −3 or −5?
−3
They're equal
−5
It depends on the context
Show the answer
−3
−3 is to the right of −5 on the number line, so it's bigger. That's why multiplying by −1 flips an inequality: 3 < 5, but −3 > −5.
A threshold sorts numbers. In a decision, one inequality splits all numbers into two groups: those that pass and those that don't. A spam filter, a pass mark and a model's "yes" all work this way. Watch five model scores sorted by the rule score ≥ 0.5.
On the number line, the marker stands at the threshold, and the shaded band running right from it is the yes side. Each step tests one score and says which side it lands on: a yes stays a large dot, a no shrinks to a small gray one. One score sits exactly on the marker: watch which side it's counted on.
Try it now
Then set the threshold to 0.7 and press Use these numbers: a stricter threshold says yes less often.
Symbols and formula. Distance from zero, written |x|, gives a neat way to state a range:
∣x∣<k⟺−k<x<k
In plain words: "x is less than k away from zero" means x is between −k and k.
So |x − 2| < 3 means x is less than 3 away from 2: between −1 and 5.
And, or, not. Conditions combine. "x > 0 and x < 10" needs both. "x < 0 or x > 10" needs at least one. "not x > 10" means x ≤ 10.
Quick check
Solve 3x − 4 ≥ 11.
x ≤ 5
x ≥ 7/3
x ≥ 5
x > 5
Show the answer
x ≥ 5
Add 4 to both sides: 3x ≥ 15. Divide by 3, a positive number, so the sign stays: x ≥ 5.
Quick check
Which range is |x − 2| < 3?
−1 < x < 5
−3 < x < 3
−5 < x < 1
x < 5
Show the answer
−1 < x < 5
x is less than 3 away from 2: from 2 − 3 = −1 to 2 + 3 = 5, ends not included.
In your own words
In your own words: why does multiplying both sides by a negative number flip an inequality?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: decisions
Purpose. In code, a comparison is a question with a True or False answer, and if acts on it.
It runs right here in your browser, with nothing to install.
The line after if runs only when the condition is True; the else part runs otherwise. Both are indented. Python's comparison signs are <, <=, >, >=, == (equal) and != (not equal).
Conditions combine with and, or and not, and a range can be chained, just as in math:
x = 0.4print(0 <= x <= 1, x > 0and x < 0.3, x < 0or x > 0.3, not x > 1)
More than two outcomes: elif.elif, short for "else if", tests another condition when the ones above were False. Python checks them top to bottom and runs the first that's True:
A named mistake: = in a condition.if x = 5: is an error. One = stores a value; a question needs ==: if x == 5:. The order of the elifs matters too: test the highest bar first, or 95 would stop at "score ≥ 70".
Now step through grade for two scores, one line at a time. Watch which lines Python visits: it tries each condition from the top, runs the first one that's True, and skips everything below it.
Worked example
Worked example: clip a number into a range
Write clip(x, lo, hi): numbers below lo become lo, numbers above hi become hi, and the rest pass through.
defclip(x, lo, hi):
if x < lo:
return lo
if x > hi:
return hi
return x
print(clip(7.5, -5, 5), clip(-12, -5, 5), clip(2, -5, 5))
In the lesson you can change this and run it.
Each if returns as soon as it's True, so the last line runs only for numbers already inside the range. 7.5 is above 5, so it comes back as 5; −12 is below −5, so it's −5; 2 is inside, so it's unchanged. Clipping keeps a number within safe limits whatever comes in, which is exactly how a training step is kept from growing too large.
89.5 is below 90, so the first test fails, but it's at least 80: B. 90 passes the first test: A.
Quick check
Which line is True exactly when x is between 0 and 1, both ends included?
x = 0 and x = 1
0 <= x <= 1
x >= 0 or x <= 1
0 < x < 1
Show the answer
0 <= x <= 1
A chained comparison with <= includes both ends.
Three ways to say one range. A range can be written as an inequality (0 ≤ x < 10), drawn on a number line (a filled dot at 0, an open dot at 10), or tested in code (0 <= x < 10). All three carry exactly the same information, and moving between them is most of the skill. The trickiest part is always the ends: say aloud whether each end is in or out, then make sure the code uses <= for "in" and < for "out".
Drill
Ranges and decisions
Fix the two functions: in_range(x, lo, hi) should be True when lo ≤ x ≤ hi, and sign_word(x) should return "negative", "zero" or "positive".
defin_range(x, lo, hi):
return lo < x < hi
defsign_word(x):
if x < 0:
return"negative"elif x > 0:
return"positive"else:
return"positive"print(in_range(5, 0, 5), sign_word(0))
In the lesson, your code is checked as you write it.
In your own words
Why does the order of the elif tests matter in grade(score)?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: decisions inside a model
Purpose. Three inequalities appear in almost every model: a threshold, ReLU and a clip.
The map below is the same in every lesson of this course: a model learning, with the parts this lesson's math lives in lit.
A threshold turns a score into a decision. A model outputs a score between 0 and 1, and "yes" means score ≥ 0.5. Where to set the threshold is a choice, and you'll try it with a widget near the end of this part.
ReLU, the most common activation in neural networks, is a function defined in two pieces:
ReLU(x)={0xif x≤0otherwise
In plain words: negative numbers become 0, and everything else passes through unchanged.
defrelu(x):
if x <= 0:
return0return x
print(relu(-3), relu(0), relu(2.5))
Drawn as a graph, ReLU is flat along 0 for every negative input, then rises as the line y = x. The widget writes the rule as max(0, x), the bigger of 0 and x, which is the same two pieces in one name.
Quick check
What does this print?
deflabel(s, t):
if s >= t:
return"yes"return"no"print(label(0.62, 0.5), label(0.62, 0.7))
yes yes
no yes
yes no
no no
Show the answer
yes no
0.62 is at least 0.5, so yes; it's below 0.7, so no. The same score, two thresholds, two answers.
Clipping and closeness. Training adjusts each weight by a step, and a rare huge step can wreck a model. So steps are often clipped to a range like −5 to 5, exactly the clip function above. And because floats are stored a hair off, code tests whether two losses are "the same" with a tolerance, not ==:
a = 0.1 + 0.2
b = 0.3print(abs(a - b) < 1e-6)
That's |a − b| < 10⁻⁶: the two are within a millionth of each other, which is what math.isclose checks too.
In the real tools. In PyTorch, the library most models are built with, each of these decisions is one call, and it runs on a whole row of numbers at once:
Two kinds of mistake. A threshold trades one mistake for another: a false alarm, a yes that should have been a no, and a miss, a no that should have been a yes. Set it low and the model says yes to nearly everything: it misses little, but raises many false alarms. Set it high and it rarely raises a false alarm, but it misses real cases. Which matters more depends on the job: a spam filter can let a few through, while a medical screen mustn't miss. Choosing the threshold is a decision about mistakes, written as one inequality.
Try it on a model that scores the tent shop's customers on whether they'll come back next season. Each dot is a customer, placed by its score, and the table counts the four outcomes. The false alarms are in the row "Really stays away", column "Said comes back"; the misses are in the row "Really comes back", column "Said stays away".
Drill
Decisions inside a model
Fix relu(x) and label(score, t), then set a and b to the labels for a score of 0.55 at thresholds 0.5 and 0.6.
defrelu(x):
if x < 0:
return x
return0deflabel(score, t):
if score > t:
return"yes"return"no"
a = label(0.55, 0.5)
b = label(0.55, 0.5)
print(relu(-2), relu(3), a, b)
In the lesson, your code is checked as you write it.
In your own words
What happens to a model's answers when its threshold moves from 0.5 up to 0.7?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write