Make it better

You have a working RAG system. Most of the quality now lives in retrieval: if the right passage reaches the model, you get a good answer; if it does not, no amount of prompting saves you. So this lesson is mostly about retrieving better. Change one thing at a time, and measure.

The levers, roughly in order of payoff

• The embedding model. This is the biggest single lever. Swapping all-MiniLM-L6-v2 for a stronger model (the BGE or E5 families) often lifts retrieval noticeably. Compare candidates on the MTEB leaderboard, and remember: change the model and you must re-embed the whole corpus.
• Chunking. Revisit size and overlap, and try to respect structure: keeping a heading with its section, or splitting on paragraphs rather than a blind word window, tends to help. Bad chunks are a common hidden cause of bad answers.
• Reranking. Retrieve more candidates than you need (say the top 20), then use a reranking model (a cross-encoder) to reorder them and keep the best four. Rerankers read the query and passage together, so they judge relevance more accurately than the first-pass vector search, at the cost of a little speed.
• Hybrid search. Vector search is great at meaning but can miss exact terms, product codes, or rare names. Hybrid search combines it with old-fashioned keyword search (BM25) and merges the results, giving you the best of both.
• More signal in the prompt. Pass a little metadata with each passage (its title or source), and tune your top-k. Sometimes the fix is simply retrieving one more passage.

Measure, do not guess

It is tempting to eyeball a few answers and call it improved. Resist that. Put together a small set of questions with known good answers, and check how often the right passage is retrieved before and after each change. That single habit will save you from endless fiddling. When you are ready for something more rigorous, retrieval benchmarks like BEIR and the MTEB suite are the standard yardsticks, and the directory lists RAG-specific evaluation datasets.

Grow up the storage

Our numpy store is perfect for learning and fine for tens of thousands of chunks. When you outgrow it, move to a real vector database for approximate nearest-neighbour search at scale, persistence, and metadata filtering. You already understand exactly what it does, so adopting one is a small step. The directory lists the popular options.

Where to go next

• Reach for a framework now, if you like. LangChain and LlamaIndex automate the pipeline you just built by hand. The difference is that you now know what every part is doing, so their abstractions will read as convenience, not mystery.
• Get better data. Retrieval can only surface what you feed it. Browse the RAG Repo directory for corpora, and mind the licence on anything you build a product from.
• Keep the concepts handy. The guides and glossary cover the ideas here in more depth.

Well done

You started with a folder of text and finished with a system that answers questions from it, grounded in real passages, with no framework doing the work for you. That understanding is the thing that makes every RAG tool you touch from here make sense. Now go point it at something you care about.