Lesson 58 showed a matrix as a movement of the plane. Two questions follow. How much does the movement grow or shrink areas? And can it be undone, so every point can be sent back where it came from? One number answers both, the determinant. It's the area factor, and when it's 0 the plane has been squashed flat onto a line, which no matrix can unsquash. When it isn't 0, the undoing matrix, the inverse, exists, and it's how a system of equations gets its one answer. In ML, a zero determinant is the warning sign of two features that say the same thing, which leaves a model with no single best answer.
What you'll be able to do
Compute a 2 × 2 determinant, ad − bc, and see it as the area of the parallelogram the unit square lands on
Know that a determinant of 0 means the plane is flattened and nothing can undo it
Find a 2 × 2 inverse by hand, with np.linalg.inv and with torch.linalg.inv, and check that A⁻¹A = I
Spot a redundant feature from a zero determinant: a problem with no single answer
Quick check
Have a guess before you start. A matrix doubles every x and triples every y. A square of area 1 is moved by it. What is its new area?
5
1
9
6
Show the answer
6
The square becomes a 2-by-3 rectangle: area 2 × 3 = 6.
The math: the area factor, and undoing
Purpose. Can a movement be undone? First, how much does it scale area?
The determinant of a 2 × 2 matrix is one number:
det(acbd)=ad−bc
In plain words: multiply down the main diagonal, multiply across the other diagonal, and subtract.
It's the factor by which the matrix scales every area. diag(3, 2) has determinant 3 × 2 − 0 × 0 = 6, so the unit square becomes a rectangle of area 6. A shear, [[1, 1], [0, 1]], has determinant 1: it slants squares but keeps their area. A rotation also has determinant 1.
Quick check
From lesson 58: what does [[1, 0], [0, −1]] do to the plane?
Rotates it a quarter turn
Shears it
Doubles it
Flips it across the x-axis
Show the answer
Flips it across the x-axis
It keeps x and negates y: a mirror. Its determinant is −1: areas keep their size, and the minus sign says the plane was flipped over.
A negative determinant means the plane was flipped over, like a page turned face down; its size still tells the area factor. A zero determinant means areas become 0: the whole plane is squashed onto a line or a point.
The picture. Watch [[2, 1], [1, 3]] two ways at once: as boxes of numbers, and as a drawing on a grid. The drawing starts from the unit square, 1 by 1, with two of its sides drawn as the unit arrows: the thick one along x, the thin one along y. The first step shows what the main diagonal alone would do, stretch the square 2 across and 3 up. The second slants the arrows onto A's columns, (2, 1) and (1, 3), where lesson 58 showed the unit arrows land, and the square lands on a parallelogram. Its area is the determinant. The last step runs the inverse, which sends the parallelogram home.
Try it now
Then set the matrix to 1 2 on the first line and 2 4 on the second, and press Use these numbers: the second row is twice the first, both arrows land on one line, the determinant is 0, and there's no inverse.
Symbols and formula. The inverse A⁻¹ undoes A: applying A and then A⁻¹ puts every point back.
A−1A=AA−1=I(acbd)−1=ad−bc1(d−c−ba)
In plain words: swap a and d, flip the signs of b and c, and divide everything by the determinant.
For [[2, 1], [1, 3]]: swap to [[3, −1], [−1, 2]] and divide by 5, giving [[0.6, −0.2], [−0.2, 0.4]]. When the determinant is 0 the formula would divide by 0, and no inverse exists. Such a matrix is called singular.
Quick check
What is the determinant of [[4, 7], [2, 6]]?
−10
38
10
24
Show the answer
10
4 × 6 − 7 × 2 = 24 − 14 = 10.
Undoing solves equations. The system 2x + y = 5 and x + 3y = 10 is A times (x, y) = (5, 10), with A = [[2, 1], [1, 3]]. Multiply both sides by A⁻¹ and the unknowns stand alone: (x, y) = A⁻¹(5, 10) = (0.6 × 5 − 0.2 × 10, −0.2 × 5 + 0.4 × 10) = (1, 3). Check: 2 + 3 = 5 and 1 + 9 = 10. When A is singular, a system like this has either no answer or endless answers, never exactly one (lesson 60).
In your own words
In your own words: why can't a matrix with determinant 0 be undone?
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: np.linalg.det and np.linalg.inv
Purpose. Let NumPy compute both, and check the result.
It runs right here in your browser, with nothing to install.
np.linalg.det prints 5.000000000000001: it computes the determinant a general way that leaves float noise, so round it or compare with np.allclose. The last line is the check that matters: A times its inverse should be the identity.
Code flow. NumPy hides the work, so step through it done by hand, one line at a time: the determinant from the two diagonals, then the inverse by swapping, flipping and dividing, then the check. A[0, 0] is the entry in row 0, column 0 (lesson 53). The arrays show as boxes, like the flow's, rounded to four places, so check looks exactly like the identity. The last line prints check[0, 1] in full: not 0, but a tiny negative number in e-notation, a hair away from it. That's why the same line asks np.allclose and not ==.
A named mistake: inverting a singular matrix.np.linalg.inv on [[1, 2], [2, 4]] can't work. On a full install it stops with "LinAlgError: Singular matrix"; the NumPy in this browser hands back a matrix of nan ("not a number") instead. Either way, check the determinant first, with a small tolerance, because float noise can leave a "zero" determinant as a tiny nonzero number:
import numpy as np
S = np.array([[1, 2], [2, 4]])
print(abs(np.linalg.det(S)) < 1e-9)
Solving a system.np.linalg.solve(A, b) finds x with A x = b directly, which is faster and more accurate than computing the inverse first. For our system it gives (1, 3).
Worked example
Worked example: invert and verify three matrices
For each matrix, check the determinant; if it isn't zero, invert and confirm A⁻¹A = I.
import numpy as np
mats = [np.array([[4, 7], [2, 6]]), np.array([[3, 1], [5, 2]]), np.array([[1, 2], [2, 4]])]
for A in mats:
d = np.linalg.det(A)
ifabs(d) < 1e-9:
print(A.tolist(), "singular: no inverse")
else:
A_inv = np.linalg.inv(A)
print(A.tolist(), round(d, 6), np.allclose(A_inv @ A, np.eye(2)))
In the lesson you can change this and run it.
The first two have determinants 10 and 1, so each has an inverse, and multiplying gives back the identity. The third has rows that repeat each other (row 2 is twice row 1), its determinant is 0, and the code reports it instead of letting np.linalg.inv fail. Checking first, with a tolerance, is the safe pattern.
Quick check
What does this print?
import numpy as np
A = np.array([[2, 1], [1, 3]])
x = np.linalg.solve(A, np.array([5, 10]))
print(np.round(x, 6).tolist())
[5.0, 10.0]
[1.0, 3.0]
[2.5, 3.33]
[3.0, 1.0]
Show the answer
[1.0, 3.0]
x = 1 and y = 3 solve 2x + y = 5 and x + 3y = 10.
Determinants multiply. Chaining two movements multiplies their area factors: det(SH) = det(S) × det(H). A stretch with determinant 6 followed by a shear with determinant 1 scales areas by 6 overall. It also explains why an inverse has determinant 1 ÷ det(A): A⁻¹A = I, and the identity's determinant is 1.
Quick check
What does this print?
import numpy as np
S = np.array([[2, 0], [0, 3]])
H = np.array([[1, 1], [0, 1]])
print(round(np.linalg.det(S), 6), round(np.linalg.det(H), 6), round(np.linalg.det(S @ H), 6))
5.0 1.0 5.0
6.0 1.0 6.0
6.0 1.0 7.0
6.0 2.0 12.0
Show the answer
6.0 1.0 6.0
det S = 2 × 3 = 6, a shear's determinant is 1, and determinants multiply: 6 × 1 = 6.
Drill
An inverse that checks first
Fix safe_inv(A) so it returns None when the determinant is 0 (within 1e-9), and otherwise the inverse. Then set ok to whether safe_inv(B) @ B is the identity.
import numpy as np
defsafe_inv(A):
return np.linalg.inv(A)
B = np.array([[4, 7], [2, 6]])
ok = Falseprint(safe_inv(B), ok)
In the lesson, your code is checked as you write it.
In your own words
Why compare the determinant with a tiny number like 1e-9 instead of with exactly 0?
Write a sentence or two, then compare it with a good answer in the lesson.
In AI and ML: a zero determinant means no single answer
Purpose. Know when a problem has exactly one answer.
Fitting a model's weights often comes down to solving a system built from the data matrix X, through the square matrix XᵀX (lesson 101). If XᵀX is singular, there's no single best set of weights. That happens when one feature repeats another: say, one column is a length in meters and another the same length in centimeters.
det(XTX)=0⟺a feature is redundant
In plain words: if one column of X is a copy, or a multiple, of another, XᵀX squashes space flat, and the weights can't be pinned down.
In X the second feature is always twice the first, and XᵀX is singular. In X2 neither feature is a multiple of the other, and the determinant is 59.
Quick check
Why is there no single best weight for each feature when feature 2 is always twice feature 1?
Weight can move between the twin features freely
There are too many examples
The features are too small
The determinant is negative, so it flips
Show the answer
Weight can move between the twin features freely
w₁x + w₂(2x) = (w₁ + 2w₂)x, so only w₁ + 2w₂ matters: endless pairs give the same predictions.
What libraries do about it. Real data rarely makes a determinant exactly 0, but near-copies make it tiny, and then the weights swing wildly with small changes in the data. Two fixes are common: drop one of the twin features, or add a small penalty on the weights (regularization), which makes the matrix comfortably invertible again. Both start from noticing that the determinant is near zero.
In PyTorch. The same check in the real tool, then the weights themselves. torch.linalg has the three calls of this lesson: det, inv and solve. Here y holds a target for each example, made as 2 × feature 1 + 1 × feature 2, and the best weights solve the system XᵀX w = Xᵀy (lesson 101 shows where it comes from). torch.round(…, decimals=4) tidies float noise, like np.round.
import torch
X2 = torch.tensor([[1.0, 2.0], [2.0, 1.0], [3.0, 5.0]])
y = torch.tensor([4.0, 5.0, 11.0])
G = X2.T @ X2
print(G.tolist(), round(torch.linalg.det(G).item(), 4))
w = torch.linalg.solve(G, X2.T @ y)
print(torch.round(w, decimals=4), torch.round(torch.linalg.inv(G) @ X2.T @ y, decimals=4))
The first line is the same XᵀX and the same determinant, 59, that the NumPy cell printed. It isn't 0, so the weights have one best value, and solve finds it: 2 and 1, the recipe y was made from. The inverse gets the same answer the long way; like np.linalg.solve, torch.linalg.solve is faster and more accurate, so it's the one to use.
Drill
Find the redundant feature
Three data tables are in tables. Set flags to a list with True for each table whose XᵀX is singular (determinant within 1e-9 of 0), in order.
import numpy as np
tables = [np.array([[1.5, 4.5], [2.2, 6.6], [3.1, 9.3]]),
np.array([[1, 0], [0, 1], [1, 1]]),
np.array([[2, 5], [4, 1], [1, 3]])]
flags = []
for X in tables:
flags.append(np.linalg.det(X) == 0)
print(flags)
In the lesson, your code is checked as you write it.
In your own words
A data set has a column of prices in dollars and another of the same prices in cents. What would you do before fitting a model, and why?
Write a sentence or two, then compare it with a good answer in the lesson.
31Formulas with fractions: clear them firstFreeClearing denominators, cross-multiplying, a letter in the bottom, and reciprocal formulas like 1/R = 1/R₁ + 1/R₂.35 min
37Inverse functions, lines and scalingFreeUndoing a function by making x the subject, domains that allow it, line forms, and unscaling predictions.40 min
38Sequences and sums: find n, a, d and rFreeEvery letter of the arithmetic and geometric formulas as the subject, and an average's window from its decay.40 min
40Review: rearrange anything (algebra)FreeBroken rearrangements to fix, mixed practice from the whole algebra block, a strategy chart, and its formula sheet.35 min
41Coordinates and distanceFreePoints as (x, y), Pythagoras, distance in any number of coordinates, and nearest neighbors.42 min
42Angles, degrees and radiansFreeMeasuring a turn, converting units, arcs, and the angle between two arrows.36 min
43Sine and cosineFreeAcross and up: right triangles, the unit circle, and directions and rotations.42 min
44Waves: height, speed and shiftFreeAmplitude, period and phase, sampling a wave, and position codes in transformers.37 min
68Vector equations: rearranging with arrowsFreeSolving for a vector or a scale, recovering a gradient from two weight snapshots, and an angle from a dot product.40 min
87Counting: choices, orders and groupsFreeThe counting principle, factorials, ordered picks and groups, and why attention's cost grows with the square of the length.43 min
88Probability: from counts to chancesFreeEqually likely outcomes, the complement, distributions, seeded simulation, and a next-word model.43 min
89Combining events: and, or, independentFreeThe or rule, independence, probability trees, drawing with and without replacement, and chains of steps.42 min
100Maximum likelihoodFreeLikelihoods as products, log-likelihoods as sums, grid search, underflow, and the negative log-likelihood loss.45 min
101Linear regression: the best lineFreeResiduals, least squares by formula, three roads to one line, the matrix form, and train versus test error.45 min
109One neuron, start to finishFreeWeighted sum, activation, loss, slopes and update, by hand and in code, and what one straight line can and can't split.45 min
111Training a small networkFreeThe backward pass in matrix form, gradient checks, a full training loop, and reading loss curves.45 min
112The same network in PyTorchFreenn.Sequential with the NumPy weights loaded, the same loss and slopes, then SGD and mini-batches.45 min
113Attention as matrix mathFreeScaled dot-product scores, the causal mask, a row-by-row softmax, and blending value vectors.45 min
114Residual connections and layer normFreeAdding the input back and why its slope keeps learning alive, layer norm by hand, and add and norm.40 min
115A transformer block, start to finishFreeA sentence through embeddings, attention, add and norm, a small network and next-word scores, every shape named.45 min
117Final test: transpose any formulaFreeSlips to fix from every topic, mixed practice of every kind of rearranging, the strategy chart and the full sheet.45 min
Learn the math behind AI from arithmetic up: numbers, algebra, vectors and matrices, calculus, probability and statistics, each shown as numbers moving and checked in Python, ending with a network you build and train from the formulas.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write