Phase 2 · Math, when you need it · Linear algebra

Dot product and cosine similarity

The one operation behind similarity search and attention.

About 25 minutes · A free account opens the whole lesson

Create a free account Try part of it here

Already have an account? Sign in to open this lesson

Why it matters

If you learn one piece of math for AI, make it this one. The dot product is how AI search ranks documents. It's how a simple scoring rule works ("2 points per page view, 10 for a demo request"). Every neuron in a neural network computes one. And attention, the heart of models like ChatGPT and Claude, is dot products between words. Learn this one move well and a lot of AI becomes familiar.

What you'll be able to do

Compute a dot product by hand, with a loop and with NumPy
Read the dot product geometrically: lengths times the cosine of the angle
Use cosine similarity to compare directions regardless of length
See a weighted score (like a lead score) as a dot product: the core of a neuron

What it covers

Start at the till
A score is a dot product
What the number tells you about direction
Cosine similarity: comparing direction only
Worked example: which product matches a shopper?
Reading a dot product correctly

Start at the till

You buy 3 coffees at $2 each and 1 sandwich at $5. The bill is:

3 × 2  +  1 × 5  =  6 + 5  =  11

You multiplied each quantity by its price, then added the results. Write the quantities as one vector, [3, 1], and the prices as another, [2, 5]. What you just did is called the dot product of the two vectors. The answer is a single number: the total.

That's the whole recipe:

Pair up the numbers in the same position.
Multiply each pair.
Add up all the products.

In plain words: the dot product multiplies matching numbers and adds the results, like working out a shopping bill.

Here it is three ways in Python: a plain loop, NumPy's function, and the @ shortcut. They all give the same answer.

import numpy as np

a = np.array([3, 1])
b = np.array([1, 2])

total = 0
for x, y in zip(a, b):
    total += x * y
print(total)            # 3×1 + 1×2 = 5
print(np.dot(a, b))     # NumPy
print(a @ b)            # the @ operator does the same for vectors

It runs right here in your browser, with nothing to install.

Go deeper How math writes it

\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{n} a_i b_i = a_1 b_1 + a_2 b_2 + \cdots + a_n b_n

Read it as: "a dot b is the sum, over every position i from 1 to n, of a's number times b's number." The big $\sum$ is the Greek letter sigma. It means "add up", and it's exactly the for loop in the code above.

Quick check

What does this print?

prices = [2, 5]
counts = [3, 1]
print(sum(p * c for p, c in zip(prices, counts)))

Show the answer

Multiply pairs and add: 2 × 3 + 5 × 1 = 11. That's your bill, and it's also a dot product.

Quick check

What is [2, 0, 3] · [4, 7, 1]?

17
[8, 0, 3]
37
11

Show the answer

2×4 + 0×7 + 3×1 = 8 + 0 + 3 = 11. The answer is one number, not a list.

A score is a dot product

Suppose a sales team scores new leads: 2 points for each page they visited, 10 points for requesting a demo, and half a point per hundred employees. Each lead is a vector of facts. The points are another vector, called the weights. The score is their dot product, exactly like the bill at the till.

import numpy as np

weights = np.array([2, 10, 0.5])          # per page, per demo request, per 100 employees
leads = {
    "Alpine Goods": np.array([6, 1, 3]),
    "Lima Running Club": np.array([3, 0, 0.4]),
    "Northwind Tech": np.array([9, 1, 12]),
}
for name, features in leads.items():
    print(f"{name:<18} score = {features @ weights}")

This is exactly what a single neuron in a neural network computes: inputs times weights, added up. The only difference is that a neural network learns its weights from examples instead of a person choosing them.

What the number tells you about direction

The dot product also says something about the arrows. Picture two arrows from the same corner:

pointing roughly the same way: the dot product is a big positive number;
at a right angle, like the corner of a page: it's exactly 0;
pointing opposite ways: it's negative.

So the sign of the dot product tells you whether two things lean together, are unrelated, or pull apart. Drag the arrow below and watch the number.

Try it now

Drag b around a. Watch the dot product shrink to 0 at a right angle and turn negative past it.

Quick check

Two arrows meet at a perfect right angle. What is their dot product?

1
It depends on their lengths
−1
0

Show the answer

At a right angle the dot product is always exactly 0, however long the arrows are.

Cosine similarity: comparing direction only

There's a catch. The dot product also grows with length. A longer arrow gets a bigger dot product with everything, even if it points the same way as a shorter one. For comparing meaning, that's unfair: a long review and a short review about the same topic should count as equally similar.

The fix is to divide by both lengths. What's left measures only direction. It's called cosine similarity, and it always lands between −1 and 1:

Cosine similarity	Meaning
close to 1	point the same way: very similar
close to 0	at right angles: unrelated
close to −1	point opposite ways: opposite

In plain words: cosine similarity = dot product ÷ (length of a × length of b). It asks "do these point the same way?" and ignores how long they are.

In the code below, b is just a made ten times longer, so it points the same way: cosine similarity 1. c is at a right angle to a: cosine similarity 0.

import numpy as np

a = np.array([2.0, 1.0])
b = 10 * a                      # same direction, ten times longer
c = np.array([-1.0, 2.0])       # at right angles to a

cos = lambda x, y: x @ y / (np.linalg.norm(x) * np.linalg.norm(y))
print("a·b =", a @ b, " cos =", round(cos(a, b), 3))
print("a·c =", a @ c, " cos =", round(cos(a, c), 3))

This is the standard way to compare embeddings in AI search. If the vectors are already normalized to length 1, cosine similarity is the dot product, because dividing by 1 changes nothing. That's why the similarity search lesson could use index @ query directly.

Go deeper Where the name "cosine" comes from

There's a second formula for the dot product, using the angle $\theta$ (theta) between the arrows:

\mathbf{a} \cdot \mathbf{b} = \|\mathbf{a}\|\,\|\mathbf{b}\| \cos\theta

$\cos\theta$ is a function from school trigonometry: it's 1 at 0°, 0 at 90° and −1 at 180°. Divide both sides by the two lengths and you get cosine similarity:

\cos\theta = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}

Another picture: the dot product is the length of a times the length of b's "shadow" on a (its projection).

Worked example

Worked example: which product matches a shopper?

A shopper's taste is [likes outdoor gear, likes fashion] = [0.9, 0.1]. Compare three products by cosine similarity and rank them.

import numpy as np

shopper = np.array([0.9, 0.1])
products = {
    "hiking boots": np.array([0.8, 0.2]),
    "silk scarf":   np.array([0.1, 0.9]),
    "rain jacket":  np.array([0.6, 0.5]),
}

def cosine(x, y):
    return x @ y / (np.linalg.norm(x) * np.linalg.norm(y))

for name, v in sorted(products.items(), key=lambda kv: -cosine(shopper, kv[1])):
    print(f"{name:<13} {cosine(shopper, v):.2f}")

In the lesson you can change this and run it.

The hiking boots point almost the same way as the shopper's taste (about 0.99), the rain jacket is in between, and the scarf points elsewhere. Recommendation systems and AI search do exactly this, just with hundreds or thousands of numbers per vector.

Quick check

Two embeddings have cosine similarity −0.05. What does that suggest?

They have the opposite meaning, strongly
They're roughly unrelated: close to perpendicular
One of them is longer
They're near-duplicates

Show the answer

They're roughly unrelated: close to perpendicular

Near 0 means close to a right angle: neither similar nor opposite.

Reading a dot product correctly

Three misreadings come up again and again.

It's one number, not a list. Adding two vectors gives you a vector. The dot product multiplies pairs and adds them all up, so it gives you a single score.

Length gets mixed in. A long document has a big vector, so it can score highly against almost anything. That's why you can't compare raw dot products between vectors of very different sizes. Cosine similarity divides the lengths away, which leaves only the direction.

A cosine of 0 isn't "opposite". A cosine of 1 means the same direction, and −1 means exactly opposite. A cosine of 0 means the two are at right angles: they have nothing to do with each other. For embeddings, that's "unrelated", not "disagrees".

Both vectors also need the same number of entries, since every number needs a partner. If NumPy complains about shapes here, one of your vectors is the wrong size.

Practice what you just read

4 code examples you can edit and run
a practice exercise that checks your code
a 5-question quiz
4 review cards for spaced repetition

Create a free account

Try a quiz question

Why is cosine similarity preferred over the raw dot product for comparing document embeddings?

It ignores length, so only direction (meaning) counts
It's faster, since it skips the multiplying
It's always positive, so scores are simpler to read
It works when the vectors have different dimensions

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Vector arithmetic and length