RAGBench
RAGBench is an explainable benchmark for retrieval-augmented generation, published by Galileo (the rungalileo organisation on Hugging Face) alongside an arXiv paper in 2024. It gathers roughly 100,000 examples by aggregating twelve existing question-answering corpora, among them PubMedQA, HotpotQA, CUAD, FinQA, TAT-QA, MS MARCO, CovidQA and TechQA, and organises them into five industry domains covering biomedical research, general knowledge, legal, customer support and finance. Each example pairs a question with its retrieved context and a generated answer, then adds the benchmark's distinctive contribution, component-level TRACe annotations. TRACe scores utilisation, relevance, adherence and completeness, which lets you measure retriever quality and generator quality separately rather than judging a pipeline only by its final answer.
Access is through the Hugging Face Hub. The data is published as Parquet files split into train, validation and test, and each of the twelve source corpora is exposed as its own named config, so you can load the whole benchmark or a single domain with the datasets library. There is no gate or application: it downloads freely, which makes it open by our access measure, and the volume is small (well under a gigabyte).
For RAG and AI work it is best used to evaluate and train grounding, faithfulness and context-relevance components, and as labelled data for an LLM-as-judge that scores those TRACe dimensions.
On licensing, take care. The dataset card states CC BY 4.0, so attribution is required, but that headline sits over twelve upstream corpora that each keep their own terms: HotpotQA is share-alike and MS MARCO restricts use to non-commercial research, among others. We therefore flag contentLicenceDiffers and treat commercial use as restricted until you check the specific component you rely on. If you need adjacent resources already listed here, see RAGTruth for span-level hallucination labels and CRAG for a factuality benchmark.
Related sources
CRAG (Comprehensive RAG Benchmark)
Meta's factual question answering benchmark of 4,409 QA pairs across five domains, paired with mock web-search and knowledge-graph retrieval APIs. It is distinct from "Corrective RAG (CRAG)", which is a retrieval method, not a dataset.
FinanceBench
A benchmark for evaluating LLMs and retrieval systems on questions about public financial filings. The full set holds 10,231 question, answer and evidence triplets; a 150 example sample is released openly.
FlashRAG Benchmark Datasets
A research toolkit that bundles standardised, pre-processed benchmark datasets for RAG evaluation across many scenarios. Every dataset comes in a single unified JSONL format, so you can swap benchmarks in and out without rewriting your data loading each time.
FRAMES (Fact, Fetch, and Reason)
A Google evaluation set of 824 challenging questions that tests factuality, multi-hop retrieval and reasoning together, each paired with a gold answer and the Wikipedia articles needed to answer it. Built to measure end-to-end RAG.