Index Ai Academy
Foundation · Foundation mathVectors and matrices

A matrix times a vector

About 42 minLooking ahead: some prerequisites aren't done yet

Why this matters

A single dot product is one weighted sum, the work of one neuron. Most of the time you want several at once: three scores for three kinds of sign, ten for ten digits, thousands for the words a language model might say next. Stack the weight vectors as the rows of a matrix, and one multiplication computes all those dot products together. That's a matrix times a vector, and it's what one layer of a neural network does to one example. This lesson does it by hand, row by row, writes it as a loop and then as one @, and uses it to make a tiny sign recognizer that scores a picture against three templates and picks the best match, first in NumPy and then as a real PyTorch layer.

  • Multiply a matrix by a vector: each row dotted with the vector gives one output
  • Check the size rule: the matrix's columns must match the vector's length, and the answer has one number per row
  • Compute A x with a loop of dot products, then with A @ x
  • Score one picture against three sign types at once, as a layer of three neurons does, in NumPy and with PyTorch's nn.Linear
Quick check
Have a guess before you start. Three neurons each take the same 25-pixel picture and give one score. How many numbers come out?

The math: one dot product per row

Purpose. Apply several weighted sums to the same vector at once.

To multiply a matrix A by a vector x, take each row of A, dot it with x, and write the result as one entry of the answer:

(1203−14)(213)=(1⋅2+2⋅1+0⋅33⋅2−1⋅1+4⋅3)=(417)\begin{pmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{pmatrix} \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix} = \begin{pmatrix} 1 \cdot 2 + 2 \cdot 1 + 0 \cdot 3 \\ 3 \cdot 2 - 1 \cdot 1 + 4 \cdot 3 \end{pmatrix} = \begin{pmatrix} 4 \\ 17 \end{pmatrix}

In plain words: row 1 times x gives the first answer, row 2 times x the second. Two rows, two answers.

The vector is written standing up, as a column, because it lines up with each row as the row slides down past it.