Length and distance: norms
Why this matters
How big is a list of numbers? There's more than one fair answer. The straight-line length of its arrow is one, the L2 norm. The distance you'd walk along a grid of streets is another, the L1 norm. And sometimes only the biggest number matters, the max norm. Machine learning uses all three: to measure errors, to keep weights small, and to stop a training step from growing too large. This lesson puts them side by side, computes each with a loop and then with one NumPy call, turns any vector into a unit vector of length 1, and shows where each norm turns up inside the losses and safety limits of real training code.
- Measure a vector's size three ways: the L2 norm, the L1 norm and the max norm
- Make a unit vector by normalizing, and find a distance as the norm of a difference
- Compute norms with a loop and with np.linalg.norm, and check that a unit vector has length 1
- See the norms inside ML: mean absolute and squared error, weight decay, and gradient clipping, in NumPy and in PyTorch
The math: how big is a vector?
Purpose. One number for the size of a whole list of numbers.
A norm measures a vector's size. Three are common:
- The L2 norm, ‖v‖₂: the arrow's straight-line length, Pythagoras in any number of dimensions. Square each component, add, take the root. For (3, −4), that's 5.
- The L1 norm, ‖v‖₁: add the sizes of the components, ignoring signs. For (3, −4), that's 7: the street-grid walk.
- The max norm, ‖v‖∞: the biggest size of any one component. For (3, −4), that's 4.
When people say "the norm" or "the length" with no label, they mean L2.