Eigenvectors (intuition)
The directions a transformation leaves pointing the same way.
Create a free account Try part of it here
Why it matters
When a matrix moves the points on a page, most arrows get knocked sideways. A few special arrows don't: the matrix only makes them longer or shorter. These are called eigenvectors, and they show what a matrix "really does". Google's original search ranking, PageRank, was one. PCA, which you'll use to shrink and plot embeddings, is built from them. You'll never need to calculate them by hand. You do need the picture, and that's what this lesson gives you.
What you'll be able to do
- Explain an eigenvector as a direction a matrix only stretches
- Check the equation Av = λv numerically
- Find the dominant eigenvector by repeatedly applying a matrix (power iteration)
- Connect eigenvectors to PageRank and to PCA
What it covers
- Arrows that keep their direction
- An example you can check
- Letting NumPy find them
- Apply it again and again
- PageRank: the web's eigenvector
- What can and can't be an eigenvector
- Where you'll see this again
Arrows that keep their direction
Imagine stretching a rubber sheet sideways. An arrow drawn pointing straight sideways only gets longer: it still points sideways. An arrow drawn straight up stays pointing up. But an arrow drawn diagonally gets dragged so it points more sideways than before. It changed direction.
The arrows that don't change direction (they only get longer, shorter, or flip around) are the matrix's eigenvectors. The amount each one is stretched by is its eigenvalue.
- eigenvalue 3: arrows that way get three times longer;
- eigenvalue 0.5: they shrink to half;
- eigenvalue −1: they flip round but stay on the same line;
- eigenvalue 0: they're squashed to nothing (the matrix squashes the page flat).
"Eigen" is German for "own": these are the matrix's own special directions.
In plain words: an eigenvector is a direction the matrix only stretches, never turns. Its eigenvalue is how much it stretches.
An example you can check
Take this matrix:
[[2, 1], [1, 2]]
Try the diagonal arrow [1, 1]. Multiply: row one gives 2×1 + 1×1 = 3, row two gives 1×1 + 2×1 = 3. The answer is [3, 3], which is the same direction, three times longer. So [1, 1] is an eigenvector with eigenvalue 3.
Now try [1, −1]: you get [1, −1] back unchanged. Another eigenvector, eigenvalue 1. Every other direction gets turned toward [1, 1]. Try it below: drag the arrow onto one of the purple lines and see it stay there.
Go deeper The defining equation
Read it as: "A times v equals lambda times v." In words: applying the matrix to v gives the same as stretching v by the number (lambda, the eigenvalue). The vector must not be all zeros.
Drag v onto a purple line and Av stays on it, stretched by λ. Off the lines, Av turns. Try Rotate 90°: no direction survives a quarter-turn, so there are no eigenvectors.
Show the answer
v is an eigenvector with eigenvalue 3
[3, 6] is 3 × [1, 2]: same direction, three times longer.
Letting NumPy find them
np.linalg.eig returns the eigenvalues and a table whose columns are the eigenvectors, each scaled to length 1. The code checks the defining idea: multiplying by the matrix gives the same as stretching by the eigenvalue.
import numpy as np A = np.array([[2.0, 1.0], [1.0, 2.0]]) values, vectors = np.linalg.eig(A) print("eigenvalues:", values) for i in range(2): v = vectors[:, i] print(v.round(3), "A @ v =", (A @ v).round(3), " λ v =", (values[i] * v).round(3))
[0.707, 0.707] is [1, 1] shrunk to length 1.
Apply it again and again
Here's why the biggest eigenvalue matters most. Start with almost any arrow and keep multiplying by the matrix. Each time, the part of the arrow along the strongest direction grows fastest. So the arrow swings around toward that direction, a bit more each step. In other words, each multiplication grows the component along the strongest direction fastest, so that component comes to dominate. (We shrink it back to length 1 after each step so the numbers don't explode.)
import numpy as np A = np.array([[2.0, 1.0], [1.0, 2.0]]) v = np.array([1.0, 0.0]) for step in range(8): v = A @ v v = v / np.linalg.norm(v) print(step, v.round(4))
It settles on [0.7071, 0.7071]: the [1, 1] direction, the one with the biggest eigenvalue. This simple loop is called power iteration. It works on enormous matrices because it only ever multiplies a matrix by a vector.
v = [1.0, 0.0] for _ in range(20): x, y = v v = [2 * x + y, x + 2 * y] size = (v[0] ** 2 + v[1] ** 2) ** 0.5 v = [v[0] / size, v[1] / size] print([round(c, 3) for c in v])
Show the answer
[0.707, 0.707]
[1, 1] is the eigenvector with the biggest eigenvalue, 3. Each step grows that part three times faster than the rest, so the arrow ends up along [1, 1]. Rescaled to length 1, that's [0.707, 0.707].
PageRank: the web's eigenvector
Picture someone browsing the web by clicking links at random, forever. Over time, which pages do they land on most? Pages that lots of popular pages link to.
Put the web's links in a big matrix: each column says how a visitor on one page spreads out over the pages it links to. The long-run share of visits per page is a list of numbers that one more click doesn't change. "Multiplying by the matrix leaves it unchanged" means it's an eigenvector with eigenvalue 1, and power iteration finds it.
That was the original idea behind Google's ranking, and it's still a good mental model for how links pass importance around a website.
Go deeper The PageRank equation
Read it as: "the link matrix M times the rank vector r gives r back": an eigenvector with eigenvalue 1.
Where you'll meet eigenvectors next
- PCA (next lesson): the eigenvectors of a dataset show the directions in which the data spreads out most. Keeping the top few is a way to shrink data while keeping its shape.
- Training: how smoothly a model learns depends on these stretch factors, which is why the bowl in the gradient descent widget makes the ball zig-zag.
Show the answer
Along the eigenvector with the biggest eigenvalue
The part along the strongest direction grows fastest, so the arrow swings toward it.
What can and can't be an eigenvector
A few facts clear up most of the confusion here.
Not every matrix has real eigenvectors. An eigenvector keeps its direction. A rotation by 90 degrees turns every arrow, so no real arrow keeps its direction. If NumPy hands you complex numbers, this is usually why.
The zero vector doesn't count. Multiplying zero by any matrix gives zero, so it would trivially qualify for everything and tell you nothing. It's excluded by definition.
np.linalg.eig returns its eigenvectors as columns. This one causes silent bugs. Reading the rows gives you numbers that look fine and are wrong. Check by multiplying: should equal .
import numpy as np A = np.array([[2.0, 0.0], [0.0, 3.0]]) values, vectors = np.linalg.eig(A) v = vectors[:, 1] # a column, not a row print(A @ v, values[1] * v)
You'll rarely compute these by hand, and you don't need to in order to use PCA or read about training. What's worth keeping is the picture: most arrows change direction under a matrix, and eigenvectors are the special ones that only stretch.
Where you'll see this again
PCA finds eigenvectors of a data table's spread, and PageRank finds one for the whole web. Libraries do the arithmetic for you, so focus on what the result means.
Practice what you just read
- 3 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 4 review cards for spaced repetition
Try a quiz question
What does an eigenvalue of −2 mean for vectors along its eigenvector?