Identity, diagonal, triangular and symmetric
Why this matters
Most matrices are just tables of numbers, but a few have patterns that make them behave simply, and they come up so often that they have names. The identity changes nothing, like multiplying by 1. A diagonal matrix stretches each coordinate by its own amount. A triangular one has zeros on one side of its diagonal, and a symmetric one is its own mirror image. Each has a job in ML: rescaling features, storing distances, and the triangular mask inside every model that writes text one word at a time, which is what stops it peeking at words it hasn't written yet. This lesson teaches you to spot each pattern, build it in NumPy, and say what it does.
- Recognize the identity, diagonal, triangular and symmetric matrices by their pattern of entries
- Predict what multiplying by I or by a diagonal matrix does, without working every entry
- Build each kind with np.eye, np.diag and np.tril, and test symmetry with np.allclose(A, A.T)
- Build the causal mask that stops a language model looking at words that come later, in NumPy and with PyTorch's torch.tril and masked_fill
The math: four patterns worth naming
Purpose. A few matrices with simple patterns, and what each one does.
The main diagonal of a square matrix runs from the top left to the bottom right: a₁₁, a₂₂, a₃₃. The patterns are all about it.
- Identity, I: 1s on the diagonal, 0s everywhere else. AI = A and IA = A: it changes nothing.
- Diagonal: anything on the diagonal, 0s off it. Multiplying a vector by it scales each coordinate by its own diagonal entry.
In plain words: D times (4, 6, 1) is (2 × 4, 0.5 × 6, 3 × 1) = (8, 3, 3). Each coordinate is scaled on its own, and none of them mix.