Store and search

A vector database can sound intimidating. At its heart, though, it is just two things: the list of vectors, and the ability to find the ones most similar to a query vector. We can build both with numpy in a few lines, and you will see there is genuinely no magic in it.

The store

Our store holds the chunks (so we can return their text and source) alongside the matrix of their embeddings:

import numpy as np

class VectorStore:
    def __init__(self, chunks, embeddings):
        self.chunks = chunks                      # list of {text, source}
        self.embeddings = np.asarray(embeddings)  # shape (N, 384), normalised

    def search(self, query_vec, k=4):
        scores = self.embeddings @ np.asarray(query_vec)   # similarity to every chunk
        top = np.argsort(-scores)[:k]                      # the k highest
        return [(self.chunks[i], float(scores[i])) for i in top]

store = VectorStore(chunks, embeddings)

How the search works

That @ is a matrix-vector multiply: it takes the dot product of the query vector against every chunk vector at once, giving a similarity score per chunk. Because we normalised our vectors in the last lesson, this dot product is the cosine similarity, a standard measure of how alike two vectors are. We then sort the scores and take the top few. This is nearest-neighbour search, done by brute force.

Let us try it. We embed a question and ask the store for the closest passages:

question = "How do I back up my database?"
query_vec = embed([question])[0]

for chunk, score in store.search(query_vec, k=3):
    print(f"{score:.3f}  {chunk['source']}")
    print(chunk["text"][:200], "...\n")

You should see the passages most related to the question float to the top, with a similarity score for each. That ranked list is the “retrieval” in Retrieval-Augmented Generation.

When to reach for a real vector database

Brute force compares the query against every chunk. That is perfectly fine up to tens of thousands of chunks, which covers a lot of real projects. Beyond that it gets slow, and you will want a proper vector database that uses approximate nearest-neighbour search to stay fast at millions of vectors, plus persistence and metadata filtering. The RAG Repo directory lists the popular ones (Chroma, Qdrant, pgvector, and friends). The good news: they all do exactly what you just built, only faster, so you now know what they are for.

Next steps

We can find the right passages. In the final build step we hand them to a language model to turn into an answer.