Perpendicular directions: bases and coordinates
Why this matters
Every vector so far was written in the usual axes: (4, 3) means 4 along x and 3 along y. Any two perpendicular directions, each 1 long, can serve as axes instead, and the same arrow then gets new numbers, its coordinates along them. Reading each one takes a single dot product. This matters because SVD and PCA, later in this module, hand back perpendicular unit directions. And researchers who study what a model has learned often find an idea stored as a direction in its vectors: how much of the idea a word has is its coordinate along that direction. This lesson tests directions, reads coordinates, rebuilds vectors from them, and takes a direction out.
- Test a set of vectors for being orthonormal: each 1 long, every pair perpendicular
- Find a vector's coordinates in an orthonormal basis with one dot product each, and rebuild the vector from them
- Say why orthonormal directions keep lengths and angles, and why reading coordinates and rebuilding undo each other
- Take one direction out of a vector, and read how much of a made-up idea each word vector holds, in NumPy and PyTorch
The math: new axes from perpendicular directions
Purpose. Describe a vector by how far it reaches along directions you choose.
The usual axes point along e₁ = (1, 0) and e₂ = (0, 1). Writing v = (4, 3) means 4 steps along e₁ plus 3 along e₂:
In plain words: a vector's numbers are amounts of the axis directions, added up.
A set of directions that builds every vector this way, each in exactly one way, is a basis (plural bases), and the amounts are the vector's coordinates in it. Any two directions that don't lie on one line make a basis of the plane. The usual axes have two extra properties: each is 1 long, and they meet at a right angle. Directions like that are orthonormal: "ortho" for right-angled, "normal" for length 1. Turn the usual axes and you get other orthonormal pairs, like q₁ = (0.6, 0.8) and q₂ = (−0.8, 0.6).