Angle, cosine similarity and projection
Why this matters
The dot product mixes two things together: how long the vectors are and how much they point the same way. Often only the direction matters. Two documents about tents should count as similar whether one is a page long and the other is a book. Dividing the dot product by both lengths removes the length and leaves the cosine of the angle between the vectors, a number from −1 to 1. That number, cosine similarity, is how search engines, recommenders and chat assistants find the text that means the same thing as your question. This lesson builds it from lessons 49 and 50, adds the projection (how far one arrow reaches along another), and ends by searching a tiny dictionary of camping words by meaning.
- Find the angle between two vectors from cos θ = a·b / (‖a‖‖b‖)
- Read cosine similarity from −1 to 1, and know that 0 means perpendicular
- Find how far one vector reaches along another: its projection
- Rank words by cosine similarity to a query, the core of search by meaning, in NumPy and with PyTorch's F.cosine_similarity and F.normalize
The math: the angle hidden in the dot product
Purpose. Compare directions regardless of length.
The dot product has a second formula, using the angle θ (theta) between the two arrows:
In plain words: the dot product is length times length times the cosine of the angle between them.
Divide both sides by the two lengths and the angle is all that's left. For a = (3, 4) and b = (4, 3): the dot product is 12 + 12 = 24, and both lengths are 5, so cos θ = 24 ÷ 25 = 0.96. The arrows lean almost together.