A vector holds one list of numbers. A matrix holds many lists at once, as a table of rows and columns. That's the shape almost all data takes before a model sees it: one row per example, one column per measurement. It's also the shape of a model's memory, since a layer's weights are a matrix too, and of a picture, whose pixels sit in rows and columns. This lesson is about reading matrices: their size, how to name one entry, and how to pull out a whole row or column, on paper and in NumPy. Getting row and column the right way around, and remembering that Python counts from 0, are the two habits that prevent most matrix bugs.
What you'll be able to do
Read a matrix's size as rows × columns, and an entry aᵢⱼ as row i, column j
See a matrix as a stack of row vectors, a row of column vectors, or a picture
Index a two-dimensional NumPy array with A[i, j], A[i] and A[:, j], counting from 0
Store a data set as a matrix X with one row per example and one column per feature, and read the same pieces from a PyTorch tensor
Quick check
Have a guess before you start. A table has 3 rows and 4 columns. How many numbers does it hold?
4
34
12
7
Show the answer
12
3 rows of 4 numbers each: 3 × 4 = 12.
The math: rows, columns and entries
Purpose. Hold many vectors together in one object.
A matrix is a rectangle of numbers. Its size is written rows × columns, always rows first:
A=(257013)
In plain words: A has 2 rows and 3 columns, so it's a 2 × 3 matrix holding 6 numbers.
Each number is an entry, named by its row and then its column: a₁₂ is row 1, column 2, which is 7, and a₂₃ is row 2, column 3, which is 3. The rule "row first, then column" never changes, and it's the one to say aloud while you point.
Quick check
From lesson 47: how many numbers are in the vector you get by unrolling a 5 × 5 sign?
55
10
5
25
Show the answer
25
Five rows of five pixels laid end to end make one vector of 25 numbers.
A sign drawn in 0s and 1s is a matrix too. Step through this 5 × 5 arrow to find entry a₂₄: first its row, then its column, then where they cross.
The picture. The flow draws the sign as a table of cells, the 1s shaded dark, so the arrow shows. Rows are numbered down the left and columns along the top, from 1, as the math counts. Each step lights what its sentence names: row 2 across, then column 4 down, then the one cell where they cross. Above the code, a PyTorch tab shows the same lines on a tensor, PyTorch's array from lesson 47: a tensor is indexed exactly like a NumPy array.
Try it now
Then set row i to 3 and column j to 1 and press Use these numbers: the left tip of the arrowhead.
Symbols and formula. A general m × n matrix and its entries:
A=a11⋮am1a12am2⋯⋯a1n⋮amn
In plain words: m rows, n columns, m × n entries; aᵢⱼ sits in row i and column j.
Two ways to read the same table. Read across and A is a stack of row vectors, (2, 7, 1) and (5, 0, 3). Read down and it's a row of column vectors, (2, 5), (7, 0) and (1, 3). Both views are useful: in a data table each row is one example, and each column is one measurement across every example.
Quick check
In A = [[2, 7, 1], [5, 0, 3]], what is a₂₁?
1
7
2
5
Show the answer
5
Row 2 is (5, 0, 3), and its first entry is 5.
Matrices built from a rule. Some matrices are defined by a formula for aᵢⱼ. The times table is aᵢⱼ = i × j: row 3, column 4 holds 12. A checkerboard is aᵢⱼ = 1 when i + j is even and 0 when it's odd, so a₁₁ = 1, a₁₂ = 0, and the 1s and 0s alternate across and down. Writing the rule is often clearer than writing every number, and it's exactly how code fills a matrix: loop over every row and column and compute each entry.
In your own words
In your own words: why does the order "row, then column" matter when naming an entry?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: two-dimensional arrays
Purpose. Store a table as one NumPy array and pull out entries, rows and columns.
A list of rows makes a two-dimensional array. .shape gives (rows, columns):
import numpy as np
A = np.array([[2, 7, 1],
[5, 0, 3]])
print(A.shape)
print(A[1, 2], A[0, 1])
print(A[1], A[:, 2])
It runs right here in your browser, with nothing to install.
A[1, 2] is row index 1, column index 2: the math's a₂₃, which is 3. A[1] is a whole row. In A[:, 2] the colon means "every row", so it's the whole of column index 2, read down: (1, 3). The colon is a slice, and it works the same way in any position.
Code flow. The same reads, one line at a time. Step through and watch the boxes under Variables: A shows as a grid of 2 rows and 3 columns, and each line after it pulls one piece out. Notice which pieces come out as a single number and which as a row of boxes.
A named mistake: counting from 1. The math numbers rows from 1 and Python from 0, so a₂₃ is A[1, 2], not A[2, 3]. On this 2 × 3 matrix, A[2, 3] stops with an IndexError, because there's no row index 2. On a bigger matrix it's worse: it quietly reads the wrong entry. Subtract 1 from both numbers, every time.
Filling a matrix by a rule.np.zeros((rows, columns)) makes a matrix of zeros to fill. Two loops, one inside the other, visit every row and every column:
import numpy as np
T = np.zeros((3, 4))
for i inrange(3):
for j inrange(4):
T[i, j] = (i + 1) * (j + 1)
print(T)
That's the times table, with + 1 turning Python's indexes back into the math's row and column numbers. The dots after each number show that np.zeros makes decimals; 12. is just 12.0.
Worked example
Worked example: rows, columns and a total
For a 3 × 2 table of daily tent rentals (one row per day, one column per shop), read its size, one entry, one day and one shop.
import numpy as np
M = np.array([[3, 8],
[6, 1],
[4, 9]])
print(M.shape, M[2, 1])
print(M[0], M[:, 1])
total = 0for i inrange(3):
total = total + M[i, 1]
print(total)
In the lesson you can change this and run it.
The table has 3 days and 2 shops, so its shape is (3, 2). M[2, 1] is day 3, shop 2: 9 tents. M[0] is the whole of day 1, (3, 8), and M[:, 1] is shop 2 across every day, (8, 1, 9). The loop walks down that column and adds it up: shop 2 rented 18 tents. Rows answer questions about one day; columns answer questions about one shop.
Quick check
What does this print?
import numpy as np
A = np.array([[4, 9, 2],
[3, 5, 7]])
print(A.shape, A[0, 2], A[:, 1])
(2, 3) 9 [3 5 7]
(2, 3) 7 [9 5]
(3, 2) 2 [9 5]
(2, 3) 2 [9 5]
Show the answer
(2, 3) 2 [9 5]
2 rows and 3 columns; row index 0, column index 2 is 2; and column index 1 read down is 9 then 5.
Quick check
Which line gives the math's a₃₁ of a NumPy matrix A?
A[0, 2]
A[3, 1]
A[:, 3]
A[2, 0]
Show the answer
A[2, 0]
Row 3 is index 2 and column 1 is index 0: subtract 1 from each.
Drill
Index and fill
Fix the three lines so a23 is the math's a₂₃ of A, col is column 1 of A (the math's first column) as a list, and board is a 4 × 4 checkerboard with 1 wherever i + j is even (counting rows and columns from 1).
import numpy as np
A = np.array([[4, 9, 2],
[3, 5, 7]])
a23 = A[2, 3]
col = A[1].tolist()
board = np.zeros((4, 4))
for i inrange(4):
for j inrange(4):
board[i, j] = 1print(a23, col, board)
In the lesson, your code is checked as you write it.
In your own words
When would you want a row of a data matrix, and when a column?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: a data set is a matrix
Purpose. Models read their data as a matrix X, and store what they learn in matrices too.
Unroll each 5 × 5 sign into 25 numbers (lesson 47) and stack the results: one row per sign, one column per pixel. That stack is the data matrix X:
X∈Rn×dn=examples,d=features
In plain words: X has one row for each example and one column for each feature, here each pixel.
Three signs of 25 pixels give a 3 × 25 matrix. X[:, 12] is the center pixel of every sign: inked in the cross and the arrow, blank in the box. A model reads columns like this to find which pixels tell the signs apart.
Quick check
A data set has 60 signs, each unrolled into 25 pixels, one row per sign. What is the shape of X?
(60, 25)
(60, 5, 5)
(85,)
(25, 60)
Show the answer
(60, 25)
One row per example and one column per feature: 60 rows, 25 columns.
Weights are a matrix too. A layer that turns 25 pixels into 3 scores, one for each kind of sign, keeps its weights in a 25 × 3 matrix W: one row per input pixel, one column per output score. Lessons 55 and 56 show how X and W meet. For now the shapes are the thing to read: every model's code is full of comments like # (60, 25), because the first question about any matrix is how many rows and how many columns it has.
Three examples of four pixels: (3, 4). Column index 2, read down, is inked in every example. Row index 1 is the second example.
In PyTorch. The data matrix as a tensor, read the same ways, and a real layer's weights. torch.from_numpy (lesson 47) turns the NumPy X into a tensor. nn.Linear(25, 3) makes a layer from 25 inputs to 3 outputs; its weights start random, and this cell only reads their shape.
import numpy as np
import torch
import torch.nn as nn
cross = np.array([[1,0,0,0,1],[0,1,0,1,0],[0,0,1,0,0],[0,1,0,1,0],[1,0,0,0,1]])
arrow = np.array([[0,0,1,0,0],[0,1,1,1,0],[1,0,1,0,1],[0,0,1,0,0],[0,0,1,0,0]])
box = np.array([[1,1,1,1,1],[1,0,0,0,1],[1,0,0,0,1],[1,0,0,0,1],[1,1,1,1,1]])
X = torch.from_numpy(np.array([cross.flatten(), arrow.flatten(), box.flatten()]))
print(X.shape)
print(X[:, 12], X[:, 0])
print(X[0, 12], X[1].shape)
layer = nn.Linear(25, 3)
print(layer.weight.shape)
The first two lines are what the NumPy cell printed, in PyTorch's words: torch.Size([3, 25]) is the shape (3, 25), and columns 12 and 0 hold the same 1s and 0s. X[0, 12] is one entry, a one-number tensor, and the row X[1] holds 25 numbers. The last line is worth a second look: PyTorch keeps a layer's weights as (out, in), here 3 × 25, one row per output score. That's the 25 × 3 W from above turned on its side, its transpose (lesson 54). So nn.Linear(25, 3) names its sizes inputs first, but its weights list the outputs first; lesson 55 shows why.
Quick check
What does this print?
import torch.nn as nn
layer = nn.Linear(4, 2)
print(layer.weight.shape)
torch.Size([2, 4])
(2, 4)
torch.Size([8])
torch.Size([4, 2])
Show the answer
torch.Size([2, 4])
nn.Linear(4, 2) has 4 inputs and 2 outputs, and PyTorch keeps its weights as (out, in): one row per output, 2 rows of 4.
Drill
One sign, one pixel
Build X from the three signs (cross, arrow, box), one unrolled sign per row. Then set arrow_row to the arrow's row as a list, corner to the top-left pixel of every sign as a list, and inked to the number of inked pixels in the box.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write