A matrix times a vector
Why this matters
A single dot product is one weighted sum, the work of one neuron. Most of the time you want several at once: three scores for three kinds of sign, ten for ten digits, thousands for the words a language model might say next. Stack the weight vectors as the rows of a matrix, and one multiplication computes all those dot products together. That's a matrix times a vector, and it's what one layer of a neural network does to one example. This lesson does it by hand, row by row, writes it as a loop and then as one @, and uses it to make a tiny sign recognizer that scores a picture against three templates and picks the best match, first in NumPy and then as a real PyTorch layer.
- Multiply a matrix by a vector: each row dotted with the vector gives one output
- Check the size rule: the matrix's columns must match the vector's length, and the answer has one number per row
- Compute A x with a loop of dot products, then with A @ x
- Score one picture against three sign types at once, as a layer of three neurons does, in NumPy and with PyTorch's nn.Linear
The math: one dot product per row
Purpose. Apply several weighted sums to the same vector at once.
To multiply a matrix A by a vector x, take each row of A, dot it with x, and write the result as one entry of the answer:
In plain words: row 1 times x gives the first answer, row 2 times x the second. Two rows, two answers.
The vector is written standing up, as a column, because it lines up with each row as the row slides down past it.