Index Ai Academy
Foundation · Foundation mathVectors and matrices

Adding and scaling vectors

About 43 minLooking ahead: some prerequisites aren't done yet

Why this matters

Vectors can be added, subtracted and stretched, and each operation has a picture: walking one arrow then another, or making an arrow longer, shorter or backward. These few moves are the whole of how a model learns. Every training step takes the weights, a vector, and subtracts a small multiple of another vector, the gradient. This lesson builds those moves by hand, in NumPy, and then as that one line of learning. Along the way come the arrow from one point to another, b − a, and the mix of two vectors, which is how a point partway between two others is found and how word vectors can be blended and compared.

  • Add and subtract vectors component by component, and picture it tail to head
  • Scale a vector by a number, to stretch, shrink or reverse it, and mix two vectors
  • Do vector arithmetic on NumPy arrays, know that sizes must match, and see why one array line beats a loop (vectorization)
  • Read a training step, w ← w − η·g, as one vector subtraction, and run it in PyTorch, by hand and with an optimizer
Quick check
Have a guess before you start. You walk (3, 1), then (1, 2). Where do you end up, starting from the origin?

The math: combining and resizing arrows

Purpose. Combine movements, and make them bigger, smaller or backward.

Adding. Add matching components: (3, 1) + (1, 2) = (4, 3). As arrows, put the second arrow's tail at the first one's head (its tip), tail to head; the sum goes from the first tail to the last head. Order doesn't matter: either way round you land in the same place.

Scaling. Multiply every component by one number, a scalar: 2 × (3, 1) = (6, 2), twice as long, same direction. A scalar between 0 and 1 shrinks it; a negative scalar reverses it: −1 × (3, 1) = (−3, −1).

Subtracting. b − a is the arrow that goes from a to b: what you'd add to a to reach b.