Index Ai Academy
Foundation · Foundation mathVectors and matrices

Eigenvectors: directions that only stretch

About 41 minLooking ahead: some prerequisites aren't done yet

Why this matters

Lesson 58 showed a matrix moving the whole plane. Watch the arrows as it moves and almost every one swings to a new direction. But a few special directions don't turn at all: arrows along them only get longer, shorter or reversed. Those are the matrix's eigenvectors, its natural axes, and the stretch factor along each is its eigenvalue. They reveal what a matrix really does once you look along the right directions. In ML they find the directions in which data spreads the most, the idea behind PCA in the next lesson, and they explain why some training runs settle smoothly while others blow up.

  • Tell an eigenvector from an ordinary vector: the matrix only stretches it, never turns it
  • Check Av = λv by hand and read the eigenvalue λ
  • Find and verify eigenvectors with np.linalg.eig, checking A @ v against λ * v for every pair
  • Find the main direction of a cloud of points as the top eigenvector of its covariance matrix, in NumPy and with torch.linalg.eigh
Quick check
Have a guess before you start. The matrix diag(3, 1) stretches x by 3 and leaves y alone. Which arrow keeps its direction?

The math: arrows that stay on their line

Purpose. Find the natural axes of a matrix.

Multiply a matrix A by a vector v and usually the result points somewhere new. Sometimes it doesn't: Av lands on the same line as v, just scaled. Then v is an eigenvector of A and the scale factor λ (lambda) is its eigenvalue:

Av=λvA\mathbf{v} = \lambda\mathbf{v}

In plain words: the matrix does to v exactly what multiplying by one number would do.

Take A = [[2, 1], [1, 2]]. Then A(1, 1) = (3, 3) = 3 × (1, 1), so (1, 1) is an eigenvector with eigenvalue 3. And A(1, −1) = (1, −1), an eigenvector with eigenvalue 1. But A(1, 0) = (2, 1), not a multiple of (1, 0): an ordinary vector, turned.