Embeddings

A practice model you can read

The academy's practice API has a tiny embedding model called gradient-embed-mini. It makes vectors of only 24 numbers. Each position, or dimension, has a name, such as footwear, outerwear, shipping or discounts. It's built from a word list rather than learned. That makes it much cruder than a real model. In return, you can see exactly why two texts come out similar.

The code sends two texts to the API and gets one vector back for each. It also asks the API for the names of the 24 dimensions. Then it prints each vector's two biggest numbers.

import requests

response = requests.post(
    "https://api.gradient.test/v1/embeddings",
    json={"input": ["waterproof coat", "free delivery on orders over $50"]},
)
data = response.json()
names = requests.get("https://api.gradient.test/v1/embeddings/info").json()["names"]

for item in data["data"]:
    vector = item["embedding"]
    top = sorted(range(len(vector)), key=lambda i: vector[i], reverse=True)[:2]
    print(item["index"], len(vector), "numbers; strongest:", [(names[i], vector[i]) for i in top])

Here's how to read the output:

• The response has one embedding per input text, in the same order. index tells you which text each one belongs to.
• Each vector has 24 numbers.
• "waterproof coat" puts all its weight on outerwear. Its second number is 0, so only one dimension counts.
• The delivery text splits its weight between shipping ("delivery") and account ("orders"). Each gets about 0.707.

Which models make embeddings

Embedding models are separate from chat models. You send text and get vectors back, not words. Anthropic doesn't offer an embedding model. Its documentation points to providers such as Voyage AI. OpenAI, Google and Cohere sell embedding models too. Open-source models, often called sentence-transformers, can run on your own computer.

Whichever you pick, two rules apply:

1. Use the same model for everything you compare. Vectors from different models live in different spaces and can't be compared.
2. Re-embed everything if you switch models. Old vectors are meaningless to the new model.

Think of shoe sizes. A US size 9 and a European size 42 are both numbers, but you can't compare them directly. With embeddings it's worse, because there's no simple conversion chart.

Measuring similarity: cosine

You have two vectors. How alike are they? The standard measure is cosine similarity. It compares the directions two arrows point in and ignores how long they are. Think of two hikers reading their compasses. You care which way each one faces, not how far each has walked.

• 1.0 means the same direction (very similar meaning)
• 0 means unrelated directions
• negative means opposite directions (rare for text)

You need two ingredients to work it out.

• The dot product: multiply the matching numbers in the two lists, then add up the results.
• The length of each vector: square each number, add the squares, then take the square root. The arrow [3, 4] is 5 long, because 3 × 3 + 4 × 4 = 25, and the square root of 25 is 5.

Cosine similarity is the dot product divided by the two lengths multiplied together.

In plain words: multiply the lists number by number and add up. Then divide by both lengths. Dividing removes the effect of length, so only direction is left.

Here's a worked example with small numbers. Take a = [1, 1, 0] and b = [1, 0, 0].

1. Dot product: 1 × 1 + 1 × 0 + 0 × 0 = 1.
2. Length of a: the square root of 1 + 1 + 0, which is the square root of 2, about 1.414.
3. Length of b: the square root of 1 + 0 + 0, which is 1.
4. Cosine: 1 ÷ (1.414 × 1) ≈ 0.707. The arrows point partly the same way.

The same steps in plain Python:

import math

a = [1, 1, 0]
b = [1, 0, 0]
dot = sum(x * y for x, y in zip(a, b))
length_a = math.sqrt(sum(x * x for x in a))
length_b = math.sqrt(sum(x * x for x in b))
print(dot, round(length_a, 3), length_b)
print(round(dot / (length_a * length_b), 3))

It prints 1 1.414 1.0, then 0.707. zip(a, b) pairs up the matching numbers.

NumPy does each step in one short piece of code. a @ b is the dot product, and np.linalg.norm(a) is the length of a:

import numpy as np

def cosine(a, b):
    a, b = np.array(a), np.array(b)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

print(round(cosine([1, 2, 0], [2, 4, 0]), 3))   # same direction, different length
print(round(cosine([1, 0, 0], [0, 1, 0]), 3))   # unrelated
print(round(cosine([1, 1, 0], [1, 0, 0]), 3))   # partly similar

It prints 1.0, 0.0 and 0.707. The first pair scores 1.0 even though the second arrow is twice as long. The last line matches the hand calculation above.

print(round(cosine([3, 4], [4, 3]), 3))

A shortcut for length-1 vectors

Most embedding APIs return vectors that are already exactly 1 long. The practice API does too. Look back at the delivery text. Its two big numbers give 0.707 × 0.707 + 0.707 × 0.707, which is about 1. The square root of 1 is 1.

When both lengths are 1, dividing by them changes nothing. So the cosine is the same as the dot product, a @ b. Skipping the division is faster when you compare thousands of vectors.

Phase 2 explains the math behind this properly. For now, "closer to 1 means more similar" is enough.

import itertools
import numpy as np
import requests

messages = [
    "Where is my parcel?",
    "My delivery hasn't arrived",
    "Can I get a refund for this jacket?",
    "I want to send my coat back",
    "Do you have this boot in size 9?",
]
vectors = np.array([item["embedding"] for item in requests.post(
    "https://api.gradient.test/v1/embeddings", json={"input": messages}).json()["data"]])

pairs = []
for i, j in itertools.combinations(range(len(messages)), 2):
    pairs.append((float(vectors[i] @ vectors[j]), messages[i], messages[j]))
for score, a, b in sorted(pairs, reverse=True)[:3]:
    print(f"{score:.2f}  {a!r}  ~  {b!r}")