Index Ai Academy
Foundation · Foundation mathVectors and matrices

Angle, cosine similarity and projection

About 45 minLooking ahead: some prerequisites aren't done yet

Why this matters

The dot product mixes two things together: how long the vectors are and how much they point the same way. Often only the direction matters. Two documents about tents should count as similar whether one is a page long and the other is a book. Dividing the dot product by both lengths removes the length and leaves the cosine of the angle between the vectors, a number from −1 to 1. That number, cosine similarity, is how search engines, recommenders and chat assistants find the text that means the same thing as your question. This lesson builds it from lessons 49 and 50, adds the projection (how far one arrow reaches along another), and ends by searching a tiny dictionary of camping words by meaning.

  • Find the angle between two vectors from cos θ = a·b / (‖a‖‖b‖)
  • Read cosine similarity from −1 to 1, and know that 0 means perpendicular
  • Find how far one vector reaches along another: its projection
  • Rank words by cosine similarity to a query, the core of search by meaning, in NumPy and with PyTorch's F.cosine_similarity and F.normalize
Quick check
Have a guess before you start. The arrows (1, 2) and (2, 4) have different lengths. How similar are their directions?

The math: the angle hidden in the dot product

Purpose. Compare directions regardless of length.

The dot product has a second formula, using the angle θ (theta) between the two arrows:

a⋅b=∥a∥ ∥b∥cos⁡θ\mathbf{a} \cdot \mathbf{b} = \lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert \cos \theta

In plain words: the dot product is length times length times the cosine of the angle between them.

Divide both sides by the two lengths and the angle is all that's left. For a = (3, 4) and b = (4, 3): the dot product is 12 + 12 = 24, and both lengths are 5, so cos θ = 24 ÷ 25 = 0.96. The arrows lean almost together.