Retrieve and generate

Everything is in place. We can turn text into chunks, embed them, and find the passages closest to a question. The final step is to hand those passages to a language model and ask it to write the answer. This is the “generation” in Retrieval-Augmented Generation.

Retrieve

First a small helper that embeds the question and returns the top passages. This is the same search from the last lesson, wrapped up:

def retrieve(question, k=4):
    query_vec = embed([question])[0]
    return store.search(query_vec, k=k)

The k is your top-k: how many passages to fetch. Too few and you may miss the answer; too many and you dilute the prompt with noise and use more of the context window. Four is a fine default to start.

Build a grounded prompt

Here is the single most important idea in RAG. We tell the model, in plain words, to answer only from the passages we give it, and to admit when the answer is not there. That instruction is what turns a confident guesser into a system that stays grounded in your documents.

SYSTEM = (
    "You answer questions using only the provided context. "
    "If the answer is not in the context, say you do not know. "
    "Cite the sources you used by their number."
)

def build_prompt(question, retrieved):
    context = "\n\n".join(
        f"[{i + 1}] (from {chunk['source']})\n{chunk['text']}"
        for i, (chunk, _score) in enumerate(retrieved)
    )
    return f"Context:\n{context}\n\nQuestion: {question}"

Generate the answer

Now the language model. We will use Anthropic’s API; set your key first with export ANTHROPIC_API_KEY=.... Any provider works the same way, so swap the client if you prefer another.

from anthropic import Anthropic

client = Anthropic()   # reads ANTHROPIC_API_KEY from the environment

def answer(question, k=4):
    retrieved = retrieve(question, k=k)
    message = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        system=SYSTEM,
        messages=[{"role": "user", "content": build_prompt(question, retrieved)}],
    )
    return message.content[0].text

A few things worth noting: the grounding instruction goes in the top-level system field, the retrieved passages and the question go in the user message, and the reply comes back as message.content[0].text.

Run it

print(answer("How do I back up my database?"))

The model reads only the passages you retrieved and answers from them, citing which ones it used. Ask it something your documents do not cover and it should tell you it does not know, rather than inventing an answer. That is the whole point.

You have built RAG

Step back and look at what just happened. A question came in, you found the most relevant passages in your own documents, and a language model wrote a grounded answer from them. Load, chunk, embed, store, retrieve, generate: that is a complete RAG system, and you built every part of it.

Next steps

It works, but it can work much better. The final lesson covers the levers that improve answer quality, and where to go from here.