Index Ai Academy
Foundation · Foundation mathVectors and matrices

Identity, diagonal, triangular and symmetric

About 37 minLooking ahead: some prerequisites aren't done yet

Why this matters

Most matrices are just tables of numbers, but a few have patterns that make them behave simply, and they come up so often that they have names. The identity changes nothing, like multiplying by 1. A diagonal matrix stretches each coordinate by its own amount. A triangular one has zeros on one side of its diagonal, and a symmetric one is its own mirror image. Each has a job in ML: rescaling features, storing distances, and the triangular mask inside every model that writes text one word at a time, which is what stops it peeking at words it hasn't written yet. This lesson teaches you to spot each pattern, build it in NumPy, and say what it does.

  • Recognize the identity, diagonal, triangular and symmetric matrices by their pattern of entries
  • Predict what multiplying by I or by a diagonal matrix does, without working every entry
  • Build each kind with np.eye, np.diag and np.tril, and test symmetry with np.allclose(A, A.T)
  • Build the causal mask that stops a language model looking at words that come later, in NumPy and with PyTorch's torch.tril and masked_fill
Quick check
Have a guess before you start. Which number leaves any number unchanged when you multiply by it?

The math: four patterns worth naming

Purpose. A few matrices with simple patterns, and what each one does.

The main diagonal of a square matrix runs from the top left to the bottom right: a₁₁, a₂₂, a₃₃. The patterns are all about it.

  • Identity, I: 1s on the diagonal, 0s everywhere else. AI = A and IA = A: it changes nothing.
  • Diagonal: anything on the diagonal, 0s off it. Multiplying a vector by it scales each coordinate by its own diagonal entry.
I=(100010001)D=(20000.50003)I = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \qquad D = \begin{pmatrix} 2 & 0 & 0 \\ 0 & 0.5 & 0 \\ 0 & 0 & 3 \end{pmatrix}

In plain words: D times (4, 6, 1) is (2 × 4, 0.5 × 6, 3 × 1) = (8, 3, 3). Each coordinate is scaled on its own, and none of them mix.