Natural Questions
Natural Questions takes genuine questions that people typed into Google and pairs each with a Wikipedia page and the passage that answers it. Because the questions come from real search behaviour rather than being written for the benchmark, they reflect how people actually ask for information.
It sits alongside MS MARCO as one of the two default choices for evaluating general-domain retrieval, and unlike MS MARCO it is released under an open, commercially usable licence. The dataset is published under CC BY-SA 3.0, so any derivative you share must carry the same licence and credit the source.
Related sources
BEIR
A collection of 18 information retrieval datasets spanning ad-hoc web search, question answering, fact verification, and duplicate question retrieval. Built by aggregating existing datasets, some originally created for other tasks and converted to retrieval format. Tests how well retrieval models generalise to unseen domains without fine-tuning, which matters for real-world RAG.
BRIGHT
A reasoning-intensive retrieval benchmark. It requires genuine reasoning to connect queries with relevant documents rather than keyword or semantic overlap, revealing model weaknesses that BEIR misses.
CoIR (Code Retrieval Benchmark)
A code retrieval benchmark spanning diverse programming tasks and languages. Built for evaluating how well models retrieve relevant source code, which matters if you are building RAG over codebases.
LongEmbed
A retrieval benchmark focused on very long documents, with an average length above 5,500 words. It addresses a gap in standard benchmarks, which mostly use short documents of at most 512 tokens (the chunks of text a model reads at once).