Module 4's slips are mostly about shape. Multiplying cells where rows should meet columns, a weight matrix stored the other way around, row 2 read where column 2 was meant, a math index used as a Python one: each either stops with a shape error or, worse, runs and gives a confident wrong answer. The cure is the habit this review practices: write down the shape of every array, check that the shapes fit before you multiply, and test new code on a case small enough to work by hand. The module's formula sheet gets a shapes column for exactly this reason. With these slips under control, the calculus of Module 5 can lean on matrices without a second thought.
What you'll be able to do
Spot the common slips of Module 4: shapes that don't fit, * for @, a transposed operand, row and column swapped, and counting from 1
Find and fix slips in matrix code that runs but gives a wrong answer
Mix problems from every lesson of the module, and run a short NumPy program in your head, line by line
Use the module's formula sheet, with a shapes column: each formula, what it says, its shapes and its Python line, and see where Modules 1 to 4 sit in a model
Quick check
Have a guess before you start. A learner computes the dot product of (1, 2, 2) and (2, 1, 2) and gets [2 2 4]. What went wrong?
The vectors have different lengths
They used *, so the products weren't added
Nothing: a dot product is a vector
They added instead of multiplying
Show the answer
They used *, so the products weren't added
a * b gives the products 2, 2 and 4 without adding them. a @ b adds them up: 8.
Find the slip
Purpose. Five slips cover most of Module 4's mistakes, and each has a quick check:
* for @.* multiplies matching cells; @ pairs rows with columns and adds. Check: a dot product is one number, and A @ x has one entry per row of A.
Shapes that don't fit. (m × n)(n × p) needs the inner sizes to match. Check: write both shapes before multiplying.
A transposed operand. A weight matrix stored as (outputs, inputs) needs .T before X @ W. Check: the inner sizes tell you which way around it must be.
Row and column swapped.A[1] is a row; a column is A[:, 1]. Check: say "row first" aloud.
Counting from 1. The math's a₂₃ is A[1, 2]. Check: subtract 1 from both numbers.
Below are thirteen worked lines, with a comment saying what each should give. Seven have a slip.
Try it now
How the checks catch them. Each slip fails a one-line test. A dot product that prints as three numbers can't be a dot product. A 2 × 3 matrix has rows 0 and 1, so A[2, 3] runs off the edge. Writing the shapes (4, 25) and (3, 25) side by side shows the inner sizes clash. Saying "row first" turns A[1] into "row 2". And 1 × 4 − 2 × 2 = 0 rules out the inverse before anything is computed. The last two fail a small case worked by hand: a table of 2 rows has ids 0 and 1, and a pattern of 3 along a list of 3 fits exactly once, not 3 − 3 = 0 times. Predict the next one before you run it.
Quick check
What does this print?
import numpy as np
a = np.array([1, 2, 2])
b = np.array([2, 1, 2])
print((a * b).tolist(), a @ b)
[2, 2, 4] 8
[2, 2, 4] [2, 2, 4]
8 [2, 2, 4]
8 8
Show the answer
[2, 2, 4] 8
a * b keeps the three products; a @ b adds them into the dot product, 8.
Slips that hide in code. The dangerous slips are the ones that run. A * B on two square matrices of the same size gives an answer of the right shape. A cosine that divides by only one length still gives a number. A function that reads A[i, j] for the math's aᵢⱼ works on every test where the matrix happens to be big enough, and quietly reads the wrong entry. The two drills below are slips of exactly this kind, and a small case worked by hand exposes each.
Quick check
W has shape (10, 64) and X has shape (32, 64). Which product gives one row of 10 scores per example?
X * W
X @ W
W @ X
X @ W.T
Show the answer
X @ W.T
(32, 64) @ (64, 10) gives (32, 10): the inner 64s meet once W is transposed.
Drill
A similarity that's too big
This cosine runs, but the cosine similarity of (3, 4) with itself comes out as 5 instead of 1. Find the slip and fix it.
import numpy as np
defcosine(a, b):
return (a @ b) / np.linalg.norm(a)
a = np.array([3.0, 4.0])
print(cosine(a, a), cosine(a, np.array([4.0, 3.0])))
In the lesson, your code is checked as you write it.
In your own words
Which known case would you use to check a cosine similarity function, and why?
Write a sentence or two, then compare it with a good answer in the lesson.
Mixed practice
Purpose. Problems from across Module 4, mixed. Name each one's lesson before you start: norms, dot products, cosines, coordinates, matrix products, determinants, rank, shapes, lookups or sliding patterns. If one stops you, the steps show the way, and the formula sheet in the last part has every rule, with its shapes, in one place.
Play computer. The program below runs the module in small: a product with * and one with @, a column, an entry named the math's way, the shape of a product with a transpose, and a sum along an axis. Each line hides one of the slips from the first part. Work out every value in your head before you type it, the way Python would print it. .tolist() and .item() turn NumPy's answers into plain Python lists and numbers, so they print plainly.
Worked example
Worked example: print the shapes, find the slip
A layer's weights arrive stored one row per output. Print the shapes, see why X @ W can't work, and fix it.
import numpy as np
X = np.array([[1, 1], [2, 0], [0, 3], [1, 2]])
W_rows = np.array([[1, 0], [0, 1], [2, 1]])
print(X.shape, W_rows.shape)
print(X.shape[1] == W_rows.shape[0])
W = W_rows.T
Z = X @ W
print(W.shape, Z.shape)
print(Z.tolist())
It runs right here in your browser, with nothing to install.
X is (4, 2): four examples, two features. The weights are (3, 2): one row per output. Written side by side, (4, 2)(3, 2) has inner sizes 2 and 3, so X @ W_rows would stop with an error, and the check prints False. Transposing gives (2, 3), the inner 2s meet, and Z is (4, 3): one row of three scores per example. Printing shapes first turns a confusing error into an obvious fix.
Quick check
What does this print?
import numpy as np
A = np.array([[4, 9, 2],
[3, 5, 7]])
print(A[1].tolist(), A[:, 1].tolist(), A[1, 2])
[3, 5, 7] [9, 5] 7
[9, 5] [3, 5, 7] 7
[4, 9, 2] [4, 3] 9
[3, 5, 7] [9, 5] 5
Show the answer
[3, 5, 7] [9, 5] 7
A[1] is the second row; A[:, 1] is the second column, read down; A[1, 2] is row 2, column 3 in the math's numbering: 7.
Quick check
Which line reads the math's entry a₃₁ of a NumPy matrix A?
A[0, 2]
A[2, 0]
A[:, 3]
A[3, 1]
Show the answer
A[2, 0]
Subtract 1 from both: row 3 is index 2, column 1 is index 0.
Drill
The wrong entry
This entry(A, i, j) should return the math's aᵢⱼ, counting rows and columns from 1. It runs on big matrices but reads the wrong entry. Find the slip and fix it.
import numpy as np
defentry(A, i, j):
return A[i, j]
A = np.array([[6, 1, 8], [7, 5, 3], [2, 9, 4]])
print(entry(A, 1, 1), entry(A, 2, 3))
In the lesson, your code is checked as you write it.
In your own words
Which kind of problem in the mixed ladder did you have to stop and think about most? What told you which method to use?
Write a sentence or two, then compare it with a good answer in the lesson.
The formula sheet
Purpose. Every formula of Module 4 in one place, with what it says, the shapes it takes, and its Python line.
lesson
formula
in plain words
shapes
in Python
48
a + b, ca
entry by entry
(n,), (n,) → (n,)
a + b, c * a
49
‖v‖ = √(Σ vᵢ²)
length
(n,) → one number
np.linalg.norm(v)
50
a · b = Σ aᵢbᵢ
pairs multiplied, then added
(n,), (n,) → one number
a @ b
51
cos θ = a·b ÷ (‖a‖ ‖b‖)
direction only
(n,), (n,) → one number
(a @ b) / (norm(a) * norm(b))
52
cᵢ = v · qᵢ, so v = Σ cᵢqᵢ
amounts along perpendicular unit directions
(n, n), (n,) → (n,)
Q.T @ v
53
aᵢⱼ
row i, then column j
(m, n)
A[i - 1, j - 1]
54
Aᵀ
rows become columns
(m, n) → (n, m)
A.T
55
(Ax)ᵢ = Σⱼ Aᵢⱼxⱼ
each row dotted with x
(m, n), (n,) → (m,)
A @ x
56
(AB)ᵢⱼ = Σₖ AᵢₖBₖⱼ
row i meets column j
(m, n), (n, p) → (m, p)
A @ B
Quick check
Which row of the sheet so far matches (5, 3) @ (3,), and what shape comes out?
Lesson 56, A @ B: shape (5, 3)
Lesson 54, A.T: shape (3, 5)
Lesson 50, a @ b: one number
Lesson 55, A @ x: shape (5,)
Show the answer
Lesson 55, A @ x: shape (5,)
A matrix times a vector: the 3s meet, and one number comes out per row, (5,).
The rest of the sheet, from determinants to tensors:
lesson
formula
in plain words
shapes
in Python
59
det = ad − bc
area factor; 0 means no inverse
(2, 2) → one number
np.linalg.det(A)
60
Ax = b
elimination, then back-substitution
(n, n), (n,) → (n,)
np.linalg.solve(A, b)
61
rank
independent columns
(m, n) → one number
np.linalg.matrix_rank(A)
62
Av = λv
only stretched
(n, n) → n pairs
np.linalg.eig(A); np.linalg.eigh(A) if A is symmetric
63
A = UΣVᵀ
pieces, biggest first
(m, n) → three arrays
np.linalg.svd(A)
64
sum along an axis
that axis disappears
(m, n) → (n,) for axis 0
X.sum(axis=0)
65
eᵢE = row i of E
a one-hot row picks a row of the table
(vocab, d), ids (n,) → (n, d)
E[ids]
66
yⱼ = w · (xⱼ, …, xⱼ₊ₖ₋₁)
a pattern slid along, a dot product at each stop
(n,), (k,) → (n − k + 1,)
x[j:j + k] @ w
Each row's shapes column is also a check: if two shapes don't fit the pattern, the slip is in that step.
Quick check
Which row of the sheet says how many outputs a pattern of 5 numbers gives as it slides along a list of 12?
Lesson 66: 12 − 5 = 7 outputs
Lesson 65: 12 × 5 = 60 outputs
Lesson 64: 12 + 5 = 17
Lesson 66: 12 − 5 + 1 = 8 outputs
Show the answer
Lesson 66: 12 − 5 + 1 = 8 outputs
A sliding pattern is a convolution: (n,) with (k,) gives n − k + 1 outputs. The pattern can start at places 0 to 7, so 8 stops.
Where Modules 1 to 4 live in a model. Module 4 is the model's skeleton. Each input becomes numbers, then a vector, and a batch of them is a matrix, one row per example. A layer is X @ W + b: the matrix product with the bias broadcast to every row. Word ids become vectors by a lookup in an embedding table, and image layers slide a small pattern along, one dot product per stop. Attention hides later words with a triangular mask, one of the special matrices. On the way back, the norm of the gradient decides when to clip it, and each update is w − η·g, vector arithmetic. Rank and the SVD read the weights themselves, which is how low-rank updates fine-tune a big layer. Modules 1 and 2 had already lit the data, the activation, the chances, the targets, the loss, the split, the metrics and the memory. The map below lights all four modules together; what's still dark comes in Modules 5 to 7.
Coming up. First, lesson 68 and lesson 69 practice rearranging vector and matrix equations. Then Module 5's calculus asks how an output changes when its inputs change a little. For a model with many weights, the answer is a vector of rates, the gradient, the same shape as the weights. Training then moves the weights a small step against it: w ← w − η g, the vector update from lesson 48. Every shape habit from this review comes along: a gradient must have exactly the shape of the weights it updates, and @, .T and axis= appear on almost every line of a backward pass.
In your own words
Pick the row of the sheet you're most likely to slip on. What check would you use to catch the slip?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write