Matrix multiplication
Many dot products at once: what a neural network layer computes.
Create a free account Try part of it here
Why it matters
Matrix multiplication is where nearly all of an AI model's computing time goes. Scoring 10,000 customers with one formula is one matrix multiplication. A neural network layer is one. Attention inside ChatGPT or Claude is a few of them. The chips that run AI (GPUs) exist mainly to do this one job fast. The good news: it's built from the dot product you already know. The one thing to watch is the shapes, because most bugs in AI code are shapes that don't fit together.
What you'll be able to do
- Multiply a matrix by a vector as a batch of dot products
- Multiply two matrices, and predict the result's shape from the inputs' shapes
- Recognize X @ W + b as a neural network layer
- Implement matrix multiplication with loops and check it against NumPy
What it covers
- Many bills at once
- Many scoring rules at once
- The shape rule: will it fit?
- A neural network layer is X @ W + b
- Matrix multiplication isn't multiplying cells
- Weights go in columns
Many bills at once
In the dot product lesson, one shopping basket times one price list gave one bill. Now imagine three customers, each with their own basket, and one price list. You want three bills.
You could do three dot products, one per customer. Matrix multiplication is just a neat way to do all of them in one go. Put the baskets in a table, one customer per row, and multiply by the price list. Out comes one bill per row.
The same works for scoring sales leads. Each row is a lead's facts; the weights are the points per fact; the result is one score per lead:
import numpy as np X = np.array([[6, 1, 3], # Alpine Goods: pages, demo requests, employees (hundreds) [3, 0, 0.4], # Lima Running Club [9, 1, 12]]) # Northwind Tech w = np.array([2, 10, 0.5]) print(X @ w) # three scores, one call
Each score is the dot product of one row with w. The first is 6×2 + 1×10 + 3×0.5 = 23.5.
In plain words: a matrix times a vector = a dot product for every row. You get one number per row.
One rule: each row must have exactly as many numbers as w has. Every fact needs a weight.
Go deeper The notation
Read it as: "X times w is a column of numbers: row 1 dotted with w, row 2 dotted with w, and so on."
Show the answer
50
One dot product per row, so one number per customer: 50.
Many scoring rules at once
Now say marketing and sales each have their own points system. Put each system's weights in a column of a second table, W. Multiplying the leads by W scores every lead with every system at once. The result is a table: one row per lead, one column per scoring rule.
Each number in the result is one dot product: a row from the first table with a column from the second.
import numpy as np X = np.array([[6, 1, 3], [3, 0, 0.4], [9, 1, 12]]) W = np.array([[2, 1], # column 0: marketing's weights [10, 25], # column 1: sales's weights [0.5, 2]]) S = X @ W print(S.shape) print(S) # row = lead, column = scoring rule
Watch the steps one multiplication at a time below. Three loops: pick a row, pick a column, then add up the products along them.
Three nested loops: pick a row i of A and a column j of B, then add up the products along k. C[0][0] = 1×5 + 2×7 = 19.
The shape rule: will it fit?
Here's the habit that prevents most AI bugs. Write the two shapes side by side:
(3 leads × 3 facts) @ (3 facts × 2 rules) → (3 leads × 2 rules)
↑ ↑
these two must match
The inside numbers must be equal: each row's facts must line up with each column's weights. The outside numbers give the shape of the answer.
| first table | second table | result |
|---|---|---|
| (1,000 leads × 3) | (3 × 2 rules) | (1,000 × 2) |
| (32 words × 768) | (768 × 3,072) | (32 × 3,072) |
| (3 × 2) | (3 × 2) | error: 2 and 3 don't match |
In plain words: (a × b) @ (b × c) gives (a × c). The middle numbers must match; they "cancel out".
Order matters too. A @ B and B @ A are usually different, and often one of them doesn't even fit.
Go deeper The formula for one entry
Read it as: "the number in row i, column j of the answer is the sum over k of row i of X times column j of W": exactly the innermost loop in the stepper.
A = (2, 3) B = (3, 4) print((A[0], B[1]) if A[1] == B[0] else "no fit")
Show the answer
(2, 4)
A[1] and B[0] are both 3, so the shapes fit, and it prints the outer sizes as a pair: (2, 4). The inner sizes must match, and the outer sizes give the result's shape.
Show the answer
No: the inside numbers, 5 and 4, don't match
(4 × 5) @ (4 × 2): the inside numbers 5 and 4 must match, and they don't.
A neural network layer is X @ W + b
A layer in a neural network does exactly this. It takes a batch of inputs X (one row per example) and multiplies by a table of learned weights W. Then it adds a row of learned starting values called the bias, b.
Each column of W is one neuron: one set of weights that turns an example into one number. A layer with 3,072 neurons has 3,072 columns.
import numpy as np rng = np.random.default_rng(0) X = rng.normal(size=(4, 3)) # 4 examples, 3 features W = rng.normal(size=(3, 5)) # 5 neurons, each with 3 weights b = np.zeros(5) H = X @ W + b print(X.shape, "@", W.shape, "->", H.shape)
The output feeds the next layer, which is another matrix multiplication, and so on. A large AI model does trillions of these multiply-and-add steps for every answer. GPUs can do thousands of them at the same moment, which is why they matter so much.
Go deeper The layer formula
Read it as: "the layer's output H is the inputs times the weights, plus the bias added to every row."
Show the answer
200 numbers: the dot product of the query with every document
(200 × 1536) times (1536,) is (200,): one similarity score per document. This is the similarity search lesson's index @ query.
Matrix multiplication isn't multiplying cells
This is the most common confusion in this lesson. NumPy has two operators, and they do very different things.
import numpy as np A = np.array([[1, 2], [3, 4]]) B = np.array([[0, 1], [1, 0]]) print(A * B) # matching cells multiplied print(A @ B) # rows against columns print(B @ A) # the other order
* multiplies matching cells. @ is the real matrix product: each answer is a row of the first table scored against a column of the second. You'll want @ whenever you see matrix multiplication in a formula.
Look at the last two results. A @ B and B @ A aren't the same. With ordinary numbers, 3 × 4 equals 4 × 3, but matrices don't work that way, so the order you write them in carries meaning.
Before you run anything, write down the shapes and check them by hand. The inner numbers must match, and the outer numbers give the answer's shape. It takes five seconds, and it catches most bugs before they happen. Experienced engineers often add a comment with the shape next to each line, as you'll see in later phases.
Weights go in columns
In a layer written as X @ W, each row of X is one example, and each column of W holds the weights for one output. So a layer from 3 features to 2 outputs has W of shape (3, 2). If you lay the weights out in rows instead, the shapes won't match, and the error can be confusing, since nothing looks wrong at first sight. When you read a formula, check which way around the authors wrote it, because some books use rows and others use columns.
Practice what you just read
- 4 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 5 review cards for spaced repetition
Try a quiz question
Which multiplication is valid?