Similarity search

Scoring one document

Each comparison gives a score: a number for how close a document is to the query. You met this score in the last lesson. It's cosine similarity, and closer to 1 means more similar.

Most embedding APIs, including the practice one, return vectors with length 1. For vectors like these, the cosine is the same as the dot product. To find a dot product, multiply the matching numbers in the two lists. Then add up the results.

Here's a small example with three numbers per vector. (The practice model uses 24, and real models use hundreds or thousands.) The query is [1.0, 0.0, 0.0] and doc 0 is [0.8, 0.6, 0.0].

• First numbers: 0.8 × 1.0 = 0.8
• Second numbers: 0.6 × 0.0 = 0
• Third numbers: 0.0 × 0.0 = 0
• Add them up: 0.8 + 0 + 0 = 0.8

So doc 0 scores 0.8 against this query.

In plain words: the score is "multiply matching numbers, then add." It comes out high when the document's numbers are big in the same places as the query's.

Scoring every document at once

You could score the documents one at a time in a Python loop. With 10,000 documents, that loop is slow. NumPy can do the whole job in one step.

First, stack the document vectors into a matrix. A matrix is a grid of numbers, like a spreadsheet. Here each row holds one document's vector. Then the @ sign multiplies the matrix by the query. This is called matrix multiplication. It takes the dot product of every row with the query, so you get one score per document. Doc 0 below is the one you scored by hand.

import numpy as np

docs = np.array([
    [0.8, 0.6, 0.0],   # doc 0
    [0.0, 0.6, 0.8],   # doc 1
    [0.6, 0.8, 0.0],   # doc 2
])
query = np.array([1.0, 0.0, 0.0])

scores = docs @ query            # one score per row
print(scores)
best = np.argsort(scores)[::-1]  # indexes, highest score first
print("ranking:", best)

Here's what happened, step by step:

• docs @ query gave [0.8 0. 0.6]. Doc 0 scores 0.8, as you worked out by hand. Doc 1 scores 0, and doc 2 scores 0.6.
• np.argsort(scores) returns positions, not scores. They're the positions that would sort the scores from lowest to highest: [1 2 0].
• [::-1] reverses that order, so the best comes first: [0 2 1].

In plain words: docs @ query means "score every document against the query." np.argsort(scores)[::-1] means "list the documents from best to worst."

import numpy as np
scores = np.array([0.3, 0.1, 0.7, 0.5])
print(np.argsort(scores)[::-1])

Top-k and a minimum score

Think of asking a friend to recommend a restaurant. You want their best two or three picks. You don't want a ranked list of every place in town. That's top-k: return only the k highest-scoring documents. A k between 3 and 10 is typical.

You also want your friend to say "I don't know" when they don't know. Say a customer asks your help bot about the weather. Without a cutoff, search still returns the least-bad article, and the bot answers from it. A minimum score fixes this. Any document scoring below it is left out, even if it made the top k. If nothing clears the bar, search returns nothing.

Here's a worked example. Five articles score 0.81, 0.64, 0.42, 0.28 and 0.05 against a query. You ask for k = 3 with a minimum score of 0.5.

1. Take the top 3 scores: 0.81, 0.64 and 0.42.
2. Drop any score below 0.5. That removes 0.42.
3. You get two articles back, scoring 0.81 and 0.64.

There's no single right minimum score. It depends on the embedding model. To find a good one, try real queries and look at the scores. Note where good matches tend to land and where bad ones land. Then set the cutoff between them.

For marketers: this is the same idea as a lookalike audience. You describe what you want as a point in a space and ask for the nearest neighbors. The minimum score is your relevance cutoff.

import numpy as np
import requests

API = "https://api.gradient.test/v1"
docs = requests.get(f"{API}/docs").json()["data"]

def embed(texts):
    data = requests.post(f"{API}/embeddings", json={"input": texts}).json()["data"]
    return np.array([item["embedding"] for item in data])

# Index once: title plus text, one row per article
index = embed([d["title"] + ". " + d["text"] for d in docs])
print("index shape:", index.shape)

def search(query, k=2, min_score=0.3):
    scores = index @ embed([query])[0]
    order = np.argsort(scores)[::-1][:k]
    return [(round(float(scores[i]), 3), docs[i]["title"]) for i in order if scores[i] >= min_score]

for q in ["when will my order arrive", "my zip broke", "is there a team discount", "what's your CEO's name"]:
    print(q, "→", search(q))