Vector databases

Exact versus approximate search

So far, every search compared the query with every stored vector. That's what index @ query does. This is called exact search, because it can't miss: it checks everything.

Exact search is fine up to tens of thousands of vectors. That covers most company knowledge bases. But the work grows with the size of the index. Ten times more vectors means ten times more arithmetic for every search.

Picture finding a friend's house in a city you don't know. Exact search is knocking on every door. It always works, but it's slow in a big city. A faster plan is to take the highway to the right part of town, then main roads, then side streets. You check only a few houses near the end.

Vector databases do the same with approximate nearest-neighbor search, or ANN for short. A common ANN index is called HNSW. It links each vector to a few of its closest neighbors, like roads between houses. A search walks along those links toward the query instead of checking everything.

The catch is in the word "approximate". The walk finds the true best matches almost every time, but not always. The share of the true best results it finds is called recall:

recall = (true top results the index found) ÷ (number of true top results)

In plain words: run an exact search to get the real best results. Count how many of them the fast index also returned. Divide by how many there were. A recall of 1.0 means it missed nothing.

Say exact search's top 5 are documents d1 to d5. The ANN index returns d1, d2, d3, d5 and d9. It found 4 of the 5, so recall is 4 ÷ 5 = 0.8. Here's the same count in Python:

true_top = ["d1", "d2", "d3", "d4", "d5"]   # exact search's top 5
ann_top = ["d1", "d2", "d3", "d5", "d9"]    # what the ANN index returned
found = sum(1 for d in ann_top if d in true_top)
print("found", found, "of", len(true_top))
print("recall:", found / len(true_top))

It prints found 4 of 5 and recall: 0.8. Real indexes usually score much closer to 1.0 than this made-up one. Their settings let you trade a little recall for more speed.

A vector store in 30 lines

The core of any vector database fits in a small class. Records live in a dictionary keyed by id. Search first keeps only the records that pass the filter. Then it builds a matrix from them and ranks them:

import numpy as np

class TinyStore:
    def __init__(self):
        self.records = {}   # id -> {"vector": ..., "metadata": ...}

    def add(self, id, vector, metadata):
        self.records[id] = {"vector": np.array(vector, dtype=float), "metadata": metadata}

    def search(self, query, k=3, where=None):
        items = [(id, r) for id, r in self.records.items()
                 if where is None or all(r["metadata"].get(key) == value for key, value in where.items())]
        if not items:
            return []
        matrix = np.array([r["vector"] for _, r in items])
        scores = matrix @ np.array(query, dtype=float)
        order = np.argsort(scores)[::-1][:k]
        return [(items[i][0], round(float(scores[i]), 3)) for i in order]

store = TinyStore()
store.add("a", [1, 0], {"lang": "en"})
store.add("b", [0.8, 0.6], {"lang": "en"})
store.add("c", [0.9, 0.44], {"lang": "de"})
print(store.search([1, 0], k=2))
print(store.search([1, 0], k=2, where={"lang": "en"}))

Here's what each part does:

• add puts a record in the dictionary under its id. Adding the same id again replaces the old record, like editing a contact.
• where is the filter, a dictionary such as {"lang": "en"}. A record passes only if its metadata matches every key and value in it. With no filter, every record passes.
• The scores are dot products, as in the similarity search lesson. np.argsort(...)[::-1] puts the best first, and [:k] keeps the top k.

store = {"a": 0.9, "b": 0.2, "c": 0.7}
top = sorted(store, key=store.get, reverse=True)[:2]
print(top)

The second search adds where={"lang": "en"}. Record c is German, so it's dropped before ranking. The result is [('a', 1.0), ('b', 0.8)].

That order matters: filter before ranking. That's how vector databases apply metadata filters. Real databases add saving to disk, ANN indexes and many users at once. Underneath, the idea is the same.

import numpy as np

reports = [
    {"id": "spring-2025", "vector": [0.9, 0.1], "year": 2025},
    {"id": "summer-2025", "vector": [0.8, 0.2], "year": 2025},
    {"id": "spring-2026", "vector": [0.7, 0.3], "year": 2026},
    {"id": "hiring-2026", "vector": [0.1, 0.9], "year": 2026},
]
query = np.array([1.0, 0.0])

def top_k(items, k):
    scores = np.array([r["vector"] for r in items]) @ query
    order = np.argsort(scores)[::-1][:k]
    return [items[i] for i in order]

rank_first = [r["id"] for r in top_k(reports, 2) if r["year"] == 2026]
filter_first = [r["id"] for r in top_k([r for r in reports if r["year"] == 2026], 2)]
print("rank, then filter:", rank_first)
print("filter, then rank:", filter_first)

Which option should you pick?

There are three common kinds of vector database:

• pgvector is an add-on for PostgreSQL, a popular database. It adds vector columns and vector search. It's a great default if you already use Postgres. Supabase includes it.
• Hosted services such as Pinecone, Weaviate, Qdrant Cloud and Turbopuffer run everything for you. There's nothing to install or keep running yourself.
• Open-source engines such as Qdrant, Milvus and Chroma can run on your own servers or on your laptop.

Whichever you choose, the vectors still come from your embedding model. If you switch to a new embedding model, you have to embed every document again.

For marketers: you probably don't need a vector database for a 200-page help center. Start with an in-memory index. Move to pgvector or a hosted service when the data, the traffic or the number of apps sharing it grows.