FlashRAG Benchmark Datasets
FlashRAG is a toolkit for running reproducible RAG experiments. Its main draw for data purposes is a large set of benchmark datasets, all converted into one consistent `JSONL` layout, where each line is a JSON record. That shared format removes the friction of every benchmark using its own schema.
The collection covers a wide range of evaluation scenarios, from open-domain question answering to multi-hop reasoning, where an answer depends on combining facts from several passages. The pre-processed datasets are hosted on Hugging Face, and the toolkit code lives on GitHub.
This is a strong pick when you want to test a system against many benchmarks under the same conditions. The toolkit code is licensed MIT, while the individual datasets keep their original licences, so review the terms for any set you plan to redistribute.
Related sources
Open RAGBench (Vectara)
A RAG evaluation benchmark from Vectara built on 1,000 arXiv papers, with multimodal content extraction and 3,045 question-and-answer pairs across scientific domains. Designed to test retrieval and answer quality on real research documents rather than short, simplified passages.
PleIAs RAG-Resources
A curated collection of datasets and resources assembled by PleIAs for building and evaluating RAG applications. A useful starting point when you want ready-made material to test retrieval pipelines without hunting for datasets one by one.
RAG-Mini-Wikipedia
A small evaluation set that pairs 918 question-and-answer pairs with a matching corpus of 3,200 Wikipedia passages. Built specifically for testing RAG systems, it is small enough to run quick evaluation loops while still covering realistic retrieval questions.