Index Ai Academy
Foundation · Foundation mathVectors and matrices

Perpendicular directions: bases and coordinates

About 35 minLooking ahead: some prerequisites aren't done yet

Why this matters

Every vector so far was written in the usual axes: (4, 3) means 4 along x and 3 along y. Any two perpendicular directions, each 1 long, can serve as axes instead, and the same arrow then gets new numbers, its coordinates along them. Reading each one takes a single dot product. This matters because SVD and PCA, later in this module, hand back perpendicular unit directions. And researchers who study what a model has learned often find an idea stored as a direction in its vectors: how much of the idea a word has is its coordinate along that direction. This lesson tests directions, reads coordinates, rebuilds vectors from them, and takes a direction out.

  • Test a set of vectors for being orthonormal: each 1 long, every pair perpendicular
  • Find a vector's coordinates in an orthonormal basis with one dot product each, and rebuild the vector from them
  • Say why orthonormal directions keep lengths and angles, and why reading coordinates and rebuilding undo each other
  • Take one direction out of a vector, and read how much of a made-up idea each word vector holds, in NumPy and PyTorch
Quick check
Have a guess before you start. Turn your map so that its "east" points along the unit arrow (0.6, 0.8). How far east does the arrow (3, 4) reach now?

The math: new axes from perpendicular directions

Purpose. Describe a vector by how far it reaches along directions you choose.

The usual axes point along e₁ = (1, 0) and e₂ = (0, 1). Writing v = (4, 3) means 4 steps along e₁ plus 3 along e₂:

v=4 e1+3 e2\mathbf{v} = 4\,\mathbf{e}_1 + 3\,\mathbf{e}_2

In plain words: a vector's numbers are amounts of the axis directions, added up.

A set of directions that builds every vector this way, each in exactly one way, is a basis (plural bases), and the amounts are the vector's coordinates in it. Any two directions that don't lie on one line make a basis of the plane. The usual axes have two extra properties: each is 1 long, and they meet at a right angle. Directions like that are orthonormal: "ortho" for right-angled, "normal" for length 1. Turn the usual axes and you get other orthonormal pairs, like q₁ = (0.6, 0.8) and q₂ = (−0.8, 0.6).