RAG-Mini-Wikipedia

RAG-Mini-Wikipedia is a deliberately small evaluation set for Retrieval- Augmented Generation, made of two matched halves: 918 questions with reference answers, and a corpus of roughly 3,200 short passages taken from Wikipedia. The passages come already split into retrieval-sized chunks, so you can load them straight into a vector database and start measuring whether your system returns the passage that actually answers each question.

It is published on the Hugging Face Hub in Parquet, an efficient columnar format that loads in seconds with the datasets library or pandas. The whole set fits comfortably in memory, so a full loop is fast: embed the 3,200 passages once, then for every question retrieve the top passages, generate an answer, and score both retrieval hit rate and answer quality. You can run all 918 pairs in minutes and repeat the run after each change to your chunking, embedding model, or prompt.

That speed is exactly what it is for. Use it to smoke-test a new pipeline end to end, to catch regressions while you iterate, or to teach the mechanics of RAG evaluation without waiting on a heavy benchmark. The questions are factual and grounded in Wikipedia, so a healthy pipeline should score well.

The trade-off is that it is a sanity check, not a verdict. The domain is narrow, the questions are relatively easy, and 918 pairs is too few to say much about how your system copes with ambiguity, long documents, or specialist topics. Read a good score as necessary but not sufficient, and move up to larger, harder benchmarks once the basics hold. The licence is simple: CC BY 3.0 permits commercial use as long as you credit the source.

For a tougher, scientific-domain test, Open RAGBench offers arXiv papers with several thousand question-answer pairs, and for broad coverage FlashRAG bundles many standard benchmarks in one format. RAG-Mini-Wikipedia is the fast first rung on that ladder.