So far a matrix has been a table, and multiplying by it a calculation. There's a second way to see it that makes much of linear algebra click: a matrix is a movement. Multiply every point of the plane by the same 2 × 2 matrix and the whole plane stretches, flips, slants or turns, with straight lines staying straight and the origin staying put. Read this way, a matrix's columns tell you at a glance what it does, two movements in a row are one matrix product, and a layer of a neural network is a movement of all its data at once, chosen during training so that the next step finds the data easier to sort.
What you'll be able to do
See a 2 × 2 matrix as a movement of the plane: a stretch, flip, shear or rotation
Read a matrix's columns as where the two unit arrows land, and write a rotation with cos θ and sin θ
Apply a matrix to many points at once, and chain two movements by multiplying their matrices
See a layer as a movement of the data that can make two groups easier to split, and move a batch of points with PyTorch's points @ M.T
Quick check
Have a guess before you start. The point (3, 1) is turned a quarter turn counterclockwise around the origin. Where does it land?
(−3, −1)
(3, −1)
(1, 3)
(−1, 3)
Show the answer
(−1, 3)
A quarter turn counterclockwise sends (x, y) to (−y, x): (3, 1) goes to (−1, 3).
The math: what a matrix does to the plane
Purpose. See what a matrix does, not just what it holds.
Multiply every point (x, y) by a 2 × 2 matrix and the whole plane moves. Four basic movements:
Stretch: [[2, 0], [0, 1]] doubles every x and keeps y, pulling the plane sideways.
Flip: [[1, 0], [0, −1]] keeps x and negates y, a mirror in the x-axis: (2, 3) goes to (2, −3).
Shear: [[1, 1], [0, 1]] adds y to x, slanting squares into parallelograms: (1, 2) goes to (3, 2).
Rotation: [[0, −1], [1, 0]] turns everything a quarter turn counterclockwise: (3, 1) goes to (−1, 3).
The origin never moves, since anything times (0, 0) is (0, 0), and straight lines stay straight.
Quick check
From lesson 55: what is [[1, 1], [0, 1]] times (1, 2)?
(3, 2)
(2, 3)
(1, 3)
(1, 2)
Show the answer
(3, 2)
Row 1: 1 + 2 = 3. Row 2: 0 + 2 = 2. The shear slid the point 2 to the right, by its height.
The picture. Here is that quarter turn applied to one point, worked as the matrix-vector product it is. The drawing shows the point (3, 1) as an arrow from the origin. Row 1 of R gives the new x: a dashed upright line that the moved point must sit on. Row 2 gives the new y, a dashed flat line, and the moved point is the arrow to where the two lines cross.
Try it now
Then set point p to 1, 0 and press Use these numbers: it lands on (0, 1), R's first column. And 0, 1 lands on (−1, 0), its second.
The columns are where the unit arrows land. Multiply a matrix by (1, 0) and you get its first column; by (0, 1), its second. So to read any 2 × 2 matrix, look at its columns: they're where the two unit arrows, the "right" arrow and the "up" arrow, end up. Every other point follows, because (x, y) is x of the first arrow plus y of the second.
Rotation by any angle. Turning the right arrow by θ lands it on (cos θ, sin θ), from lesson 43, and the up arrow on (−sin θ, cos θ). Those are the columns:
Rθ=(cosθsinθ−sinθcosθ)
In plain words: to turn by θ, build the matrix from cos θ and sin θ like this. At θ = 90°, it's [[0, −1], [1, 0]].
Two movements in a row are one matrix product. First stretch with S, then rotate with R: the point p goes to R(Sp), which is (RS)p. So the combined movement is the single matrix RS, and the order matters (lesson 56): the matrix applied first sits on the right. Stretching x then rotating sends (1, 1) to (−1, 2); rotating then stretching x sends it to (−2, 1).
What a 2 × 2 matrix can't do: shift. Moving every point 3 right and 2 down would move the origin, and a matrix never moves the origin. The trick is to add a third coordinate fixed at 1: (4, 5) becomes (4, 5, 1), and the 3 × 3 matrix [[1, 0, 3], [0, 1, −2], [0, 0, 1]] sends it to (7, 3, 1). Lesson 109 uses the same trick to fold a neuron's bias into its weights.
Quick check
A matrix has columns (2, 0) and (0, 2). What does it do to the plane?
Rotates by 90°
Doubles x and leaves y alone
Flips across the x-axis
Doubles everything, in both directions
Show the answer
Doubles everything, in both directions
The right arrow lands on (2, 0) and the up arrow on (0, 2): both twice as long, same directions.
In your own words
In your own words: why are a matrix's columns enough to know what it does to every point?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: move many points at once
Purpose. Apply a matrix to a whole shape, and chain movements with @.
Put the points side by side as the columns of a 2 × n matrix. One product moves them all:
It runs right here in your browser, with nothing to install.
The four corners of the unit square, (0, 0), (1, 0), (1, 1) and (0, 1), land on (0, 0), (0, 1), (−1, 1) and (−1, 0): the square has turned a quarter turn about the origin. Each column of the answer is one moved point.
Code flow. One product moved all four corners. To see that it's four matrix-vector products side by side, step through a loop that moves one corner at a time: square[:, c] is column c, one corner (lesson 53), and moved[:, c] = … writes the moved corner into column c of an array of 0s. Then compare with the single R @ square.
Chaining, and the order. Multiply the matrices first, applied-first on the right:
import numpy as np
S = np.array([[2, 0], [0, 1]])
R = np.array([[0, -1], [1, 0]])
p = np.array([1, 1])
print((R @ S @ p).tolist(), (S @ R @ p).tolist())
R @ S @ p stretches first, then rotates; S @ R @ p rotates first. The answers differ. A named mistake: writing the matrices in the order you do them, S @ R for "stretch then rotate". The one done first goes nearest the point, on the right.
Worked example
Worked example: rotate a shape by any angle
Build the rotation matrix for 30° and turn the three corners of a triangle with it.
import math
import numpy as np
defrotation(deg):
t = math.radians(deg)
return np.array([[math.cos(t), -math.sin(t)],
[math.sin(t), math.cos(t)]])
tri = np.array([[2, 0, 0],
[0, 2, 0]])
print(np.round(rotation(30) @ tri, 3).tolist())
print(np.round(rotation(90) @ tri, 3).tolist())
print(math.cos(math.radians(90)))
In the lesson you can change this and run it.
rotation turns degrees into radians (lesson 42) and builds R_θ from cos and sin. The triangle's corners are its columns. At 30°, (2, 0) goes to (1.732, 1): two units along the 30° line. At 90° the matrix is [[0, −1], [1, 0]] up to float noise: the last line shows cos 90° as 6.123233995736766e-17, a tiny number instead of 0, which np.round tidies to 0.
Quick check
What does this print?
import numpy as np
F = np.array([[1, 0], [0, -1]])
H = np.array([[1, 1], [0, 1]])
p = np.array([2, 3])
print((H @ F @ p).tolist())
[-1, 3]
[2, -3]
[-1, -3]
[5, -3]
Show the answer
[-1, -3]
F acts first: (2, 3) becomes (2, −3). Then H adds y to x: (2 + (−3), −3) = (−1, −3).
Quick check
Two quarter turns in a row. What does this print?
import numpy as np
R = np.array([[0, -1], [1, 0]])
print((R @ R).tolist(), (R @ R @ np.array([2, 1])).tolist())
[[0, 1], [-1, 0]] [1, -2]
[[-1, 0], [0, -1]] [2, 1]
[[1, 0], [0, 1]] [2, 1]
[[-1, 0], [0, -1]] [-2, -1]
Show the answer
[[-1, 0], [0, -1]] [-2, -1]
Two quarter turns make a half turn, [[−1, 0], [0, −1]], which sends (2, 1) to (−2, −1).
Drill
Stretch, then turn
Fix the lines so combo is the single matrix for "stretch x by 3, then rotate a quarter turn", moved is the square's corners moved by it (as a list), and side_ok checks that the moved square's first side runs from (0, 0) to (0, 3).
import numpy as np
S = np.array([[3, 0], [0, 1]])
R = np.array([[0, -1], [1, 0]])
square = np.array([[0, 1, 1, 0],
[0, 0, 1, 1]])
combo = S @ R
moved = (combo * square).tolist()
side_ok = Falseprint(combo.tolist(), moved, side_ok)
In the lesson, your code is checked as you write it.
In your own words
Why does the matrix applied first go on the right in R @ S @ p?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: a layer moves the data
Purpose. A layer moves every data point at once, so that the next step can sort them.
A layer's weight matrix is a movement like these, applied to all the examples together. Training chooses the movement. A good one carries tangled groups to places where a simple rule splits them. Here two groups of three points sit on either side of the line y = x, so no rule on x alone can separate them:
M=(11−11)M(xy)=(x−yx+y)
In plain words: the new first coordinate is x − y, which is negative for every point above the line and positive for every point below it.
import numpy as np
A = np.array([[1, 2, 3], [2, 3, 4]])
B = np.array([[2, 3, 4], [1, 2, 3]])
M = np.array([[1, -1], [1, 1]])
print(A[0].tolist(), B[0].tolist())
print((M @ A).tolist())
print((M @ B).tolist())
Each group's points are the columns. Before the move, group A's x values (1, 2, 3) overlap group B's (2, 3, 4). After it, group A's new x is −1 for every point and group B's is 1, while the new y is 3, 5 and 7 in both: "is the new x below 0?" sorts them perfectly. The picture shows the move.
Quick check
Scaling features by diag(1, 0.1) is also a movement. What does it do to the plane?
Flips it
Squashes its height to a tenth
Shifts it down by 0.1
Rotates it slightly
Show the answer
Squashes its height to a tenth
The columns are (1, 0) and (0, 0.1): the right arrow stays, the up arrow shrinks to a tenth.
In PyTorch. Real data comes the other way around: one point per row, the way a batch holds one example per row (lesson 56). So the six points are the rows of a 6 × 2 tensor, points. Moving rows needs M turned around and on the right: points @ M.T does to each row what M @ p does to a column. That's exactly what nn.Linear(2, 2, bias=False) computes: a layer with 2 inputs, 2 outputs and no bias, here loaded with M.
import torch
import torch.nn as nn
M = torch.tensor([[1.0, -1.0], [1.0, 1.0]])
points = torch.tensor([[1.0, 2.0], [2.0, 3.0], [3.0, 4.0], [2.0, 1.0], [3.0, 2.0], [4.0, 3.0]])
moved = points @ M.T
print(moved[:, 0].tolist())
layer = nn.Linear(2, 2, bias=False)
with torch.no_grad():
layer.weight.copy_(M)
print(torch.equal(layer(points), moved))
The new x column holds −1 for group A's three points and 1 for group B's, the numbers the NumPy cell printed, now as decimals. torch.equal says the layer gives exactly the same moved points: a layer's weights are a movement of the data.
Quick check
A batch of 50 points is the rows of P, a 50 × 2 tensor, and M is 2 × 2. Which line moves every point by M?
P * M
M.T @ P
M @ P
P @ M.T
Show the answer
P @ M.T
Each row p needs M p, and a row times Mᵀ gives exactly that: P @ M.T is 50 × 2, one moved point per row.
Why movements matter. One straight cut, like "is the new x below 0?", is all a single neuron can make. A layer first moves the data so that one cut is enough. Deep networks stack many such movements, with a bend between them (lesson 110), until even badly tangled groups come apart. Rescaling features, as in lesson 57, is the simplest movement of all: a diagonal one that makes every feature count about equally.
Drill
Did the move help?
Two groups of points are the columns of A and B. split_before should say whether every x of A is below every x of B, but its test is wrong: fix it. Then set newA and newB to each group's first coordinate after M, as lists, and split to the same test on them.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write