Free Python developer course
Go from your first line of code to working as a Python developer: projects, testing, data, databases, web apps and APIs, all in your browser.
Close enough for floats, seeds and what they fix, properties that hold for any input, a gradient check and overfit-one-batch.
Ordinary tests say assert f(2) == 4. Numeric code breaks that habit twice. A float result is almost never exactly what you wrote down, and a function that uses random numbers gives a different answer each run. The usual reaction is to skip the test, and then a change that quietly breaks the model reaches production. This lesson gives you the four tools that fix both: a tolerance for equality, a generator you control, properties that hold for every input, and one test that trains a tiny model on purpose.
0.1 + 0.2 == 0.3 give?False
0.1 and 0.2 have no exact binary form, so their sum is 0.30000000000000004, a hair above 0.3. Comparing floats with == is a trap. This lesson shows the fix.
This rests on Testing with pytest: a test is a function that asserts what should be true, and a fixture builds what it needs.
Numeric code asks three new questions of a test.
What does "equal" mean? A float is a rounded number. Two correct calculations of the same value can differ in the last digits, so a test compares with a tolerance: the numbers are equal if the gap between them is small. A relative tolerance (rtol) scales with the size of the numbers, and an absolute tolerance (atol) covers values near zero. The common rule is |a − b| ≤ atol + rtol × |b|.
Is it repeatable? A function that draws random numbers gives new answers every run, which makes a test that passes on Monday and fails on Tuesday. A seed starts a generator from a fixed place, so it makes the same draws every time. It fixes only the generator you seeded: a different library, the order of a set, or a thread race still varies.
What is true for any input? When there is no single right answer to write down, test a property: the output shape, a sum that must be 1, an answer that must not change when the input is shifted.
def test_double(): assert 2 * 2 == 4
pytest collects functions named test_something and runs them. A passing test returns without an exception: a failing assert raises, which pytest reports. A function that only returns a value checks nothing.
The picture. One sum, checked three ways. Step through what == sees, the size of the gap, and what a tolerance says about it.
Step through. The question is never are they identical; it is whether the gap is small enough to ignore.
Write a sentence or two, then compare it with a good answer in the lesson.
Purpose. Four tools, each a few lines: a tolerance, a generator you pass in, a property, and a gradient check.
A tolerance. np.allclose and pytest.approx both compare with the rule from above. They accept numbers, lists and arrays:
import numpy as np import pytest total = 0.1 + 0.2 print(total == 0.3, np.allclose(total, 0.3), total == pytest.approx(0.3)) print(np.allclose([1.0, 2.0], [1.0, 2.1]), np.allclose([1.0, 2.0], [1.0, 2.0000001])) print(np.allclose(0.0, 1e-12), np.allclose(0.0, 1e-12, atol=0))
The last line shows why atol exists. Next to zero the relative tolerance shrinks to nothing, so a value of 1e-12 would never be close to 0 without an absolute tolerance.
A generator you pass in. If a function draws its own random numbers, a test cannot control them. Give it the generator as an argument, and the test passes a seeded one. The production code passes a fresh one:
import numpy as np def add_noise(x, rng): return x + rng.normal(0, 0.1, size=x.shape) x = np.zeros(3) a = add_noise(x, np.random.default_rng(7)) b = add_noise(x, np.random.default_rng(7)) print(np.array_equal(a, b), a.shape)
import numpy as np near = np.allclose(100.0, 100.001, rtol=1e-3, atol=0) far = np.allclose(100.0, 100.5, rtol=1e-3, atol=0) print(near, far)
True False
The tolerance is 1e-3 × 100.001 ≈ 0.1. A gap of 0.001 is far inside it, and a gap of 0.5 is well outside. A relative tolerance grows with the size of the numbers.
Code flow. A gradient check compares a hand-derived slope with a finite difference: nudge x up and down by a tiny h and see how much the function moved. It is how you test a derivative you wrote by hand, which is what every custom layer is. For f(x) = x³ the slope is 3x², which at x = 2 is 12.
random.Random(5) inside its body. A test calls it twice. What does this print?import random def pick(items): return random.Random(5).choice(items) print(pick([1, 2, 3, 4]) == pick([1, 2, 3, 4]))
True
The generator is made fresh from the same seed on every call, so both calls make the same draw. A seed inside the function makes it repeatable, but the test cannot choose a different seed. Passing the generator in is more flexible.
Compare its gradient with a finite difference at many random inputs
A finite difference measures the slope straight from the forward function, so it catches a wrong formula anywhere in the range. Running without an error or giving the right shape says nothing about the numbers.
Write is_close(a, b, rtol=1e-5, atol=1e-8): True when abs(a - b) <= atol + rtol * abs(b). That is the rule NumPy uses.
def is_close(a, b, rtol=1e-5, atol=1e-8): ... print(is_close(0.1 + 0.2, 0.3), is_close(1.0, 1.1))
Write grad_ok(f, df, x, h=1e-5, tol=1e-6): compare the central difference (f(x + h) - f(x - h)) / (2 * h) with df(x), and return True when they differ by less than tol.
def grad_ok(f, df, x, h=1e-5, tol=1e-6): ... print(grad_ok(lambda x: x ** 3, lambda x: 3 * x ** 2, 2.0)) print(grad_ok(lambda x: x ** 3, lambda x: 2 * x ** 2, 2.0))
A property test. For softmax there is no single "right" output to write down for random inputs, but three things are true of every output. The probabilities add up to 1, the shape is unchanged, and adding the same number to every score changes nothing. The cell checks all three on made-up scores, and then a naive version on a large input, to see what a property test catches that a hand-picked example can miss:
import numpy as np def softmax(x): e = np.exp(x - np.max(x, axis=-1, keepdims=True)) return e / e.sum(axis=-1, keepdims=True) def naive(x): e = np.exp(x) return e / e.sum(axis=-1, keepdims=True) scores = np.array([[1.0, 2.0, 3.0], [0.0, 0.0, 0.0]]) probs = softmax(scores) print(np.allclose(probs.sum(axis=-1), 1.0), probs.shape == scores.shape) print(np.allclose(softmax(scores), softmax(scores + 100))) big = np.array([1000.0, 1000.0]) print(softmax(big).tolist(), np.isnan(naive(big)).any())
(NumPy may print a warning for the naive version. That is the bug being caught.) The naive version is correct on small inputs and returns nothing but NaN on large ones. A property such as "the output sums to 1, for scores of every size" finds it.
softmax([1, 2, 3]) against numbers copied from the output. What could it miss?A version that overflows on large inputs
A fixed example pins down only that one input. A naive softmax passes it and returns NaN for [1000, 1000]. A test for a property over many inputs, including large ones, catches it.
Overfit one batch. The best single test of a training loop is to train a tiny model on a handful of examples and expect the loss to fall to nearly zero. If it cannot memorize eight examples, something is broken: the loss, the gradient, the optimizer or the data. The cell fits a linear model to eight points:
import numpy as np rng = np.random.default_rng(0) X = rng.standard_normal((8, 3)) w_true = np.array([1.0, -2.0, 0.5]) y = X @ w_true w = np.zeros(3) losses = [] for step in range(200): grad = 2 * X.T @ (X @ w - y) / len(X) w -= 0.1 * grad losses.append(float(np.mean((X @ w - y) ** 2))) print(losses[0] > losses[-1], losses[-1] < 0.001)
The loss started high and ended below 0.001. The test is cheap, it runs in a fraction of a second, and it fails loudly if a change breaks the learning.
The three property tests as plain pytest functions, with the generator passed in. Run directly here, as a list of results.
import numpy as np import pytest def softmax(x): e = np.exp(x - np.max(x, axis=-1, keepdims=True)) return e / e.sum(axis=-1, keepdims=True) def test_sums_to_one(): x = np.random.default_rng(0).standard_normal((4, 6)) assert softmax(x).sum(axis=-1) == pytest.approx(1.0) def test_shift_invariant(): x = np.random.default_rng(1).standard_normal(5) assert softmax(x) == pytest.approx(softmax(x + 1000)) def test_shape_kept(): assert softmax(np.zeros((8, 5))).shape == (8, 5) for test in (test_sums_to_one, test_shift_invariant, test_shape_kept): test() print(test.__name__, "passed")
Each test builds its own seeded generator, so it repeats. The comparisons use pytest.approx, never ==. And no test needs a number copied from the output: they say what must be true, so they keep passing when the implementation changes, and fail when it breaks.
Write a sentence or two, then compare it with a good answer in the lesson.
Purpose. In a real repository these tools sit in a tests/ folder and run on every push.
With PyTorch the same ideas have their own names. torch.manual_seed(0) seeds PyTorch's generator, and a test seeds it first. torch.testing.assert_close is allclose for tensors: it checks shape, dtype and values, and its message says how far apart they are. A tiny-model fixture builds a small network with a fixed seed, so every test starts from the same weights. And a golden file stores an output you have inspected and approved, so a later change that moves it is noticed.
import torch torch.manual_seed(0) a = torch.randn(3) torch.manual_seed(0) b = torch.randn(3) print(torch.equal(a, b)) torch.testing.assert_close(torch.tensor([1.0, 2.0]), torch.tensor([1.0, 2.0000001])) print("close enough") try: torch.testing.assert_close(torch.tensor([1.0, 2.0]), torch.tensor([1.0, 2.1])) except AssertionError as error: print(str(error).splitlines()[0])
(This page runs a teaching subset of PyTorch: same calls, CPU only. The message wording matches the real library's first line.)
A flaky test, traced. A test that shuffles its data with an unseeded generator passes most days and fails some. The fix is the one from the top of the lesson: the function takes the generator, the test passes a seeded one.
import random def first_after_shuffle(items, rng): items = list(items) rng.shuffle(items) return items[0] a = first_after_shuffle([1, 2, 3, 4, 5], random.Random(0)) b = first_after_shuffle([1, 2, 3, 4, 5], random.Random(0)) print(a, a == b)
With a seed the first item is always the same, so a test can name it. Without one, any answer between 1 and 5 is possible, and the test would fail four times in five.
On the job: add three tests to every model library before anything else. A shape test for each public function, an overfit-one-batch test for the training loop, and a gradient check for every custom layer. Seed everything a test uses. When a test is flaky, look for the unseeded random call first.
Write a sentence or two, then compare it with a good answer in the lesson.
monkeypatch, the tools these tests are built from.Why does a test of a function that uses random numbers pass the generator in as an argument?
575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.
Go from your first line of code to working as a Python developer: projects, testing, data, databases, web apps and APIs, all in your browser.
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write