Index Ai Academy
Foundation · Foundation mathVectors and matrices

Length and distance: norms

About 42 minLooking ahead: some prerequisites aren't done yet

Why this matters

How big is a list of numbers? There's more than one fair answer. The straight-line length of its arrow is one, the L2 norm. The distance you'd walk along a grid of streets is another, the L1 norm. And sometimes only the biggest number matters, the max norm. Machine learning uses all three: to measure errors, to keep weights small, and to stop a training step from growing too large. This lesson puts them side by side, computes each with a loop and then with one NumPy call, turns any vector into a unit vector of length 1, and shows where each norm turns up inside the losses and safety limits of real training code.

  • Measure a vector's size three ways: the L2 norm, the L1 norm and the max norm
  • Make a unit vector by normalizing, and find a distance as the norm of a difference
  • Compute norms with a loop and with np.linalg.norm, and check that a unit vector has length 1
  • See the norms inside ML: mean absolute and squared error, weight decay, and gradient clipping, in NumPy and in PyTorch
Quick check
Have a guess before you start. To get from (0, 0) to (3, 4), how far do you walk on a grid of streets, and how far does a bird fly?

The math: how big is a vector?

Purpose. One number for the size of a whole list of numbers.

A norm measures a vector's size. Three are common:

  • The L2 norm, ‖v‖₂: the arrow's straight-line length, Pythagoras in any number of dimensions. Square each component, add, take the root. For (3, −4), that's 5.
  • The L1 norm, ‖v‖₁: add the sizes of the components, ignoring signs. For (3, −4), that's 7: the street-grid walk.
  • The max norm, ‖v‖∞: the biggest size of any one component. For (3, −4), that's 4.

When people say "the norm" or "the length" with no label, they mean L2.