BRIGHT
BRIGHT is a retrieval benchmark built to test something most benchmarks quietly skip: whether a system can find the right document when the connection between the question and the answer takes actual reasoning, not just shared keywords or similar phrasing. Its queries are drawn from reasoning-heavy domains such as economics, biology, coding, mathematics, and technical problem-solving, and the relevant document often does not restate the question at all. It might state a principle, a formula, or an example you have to reason from to see that it answers the query.
You can get the data from the project's GitHub page and Hugging Face, where it comes as query sets paired with document collections and human relevance judgments, ready to plug into standard retrieval evaluation tools. The natural way to use it is to index the documents in your vector database or search engine, run your retriever over the queries, and score how often the genuinely relevant documents come back near the top.
BRIGHT is best as a diagnostic rather than a knowledge source. If you are choosing an embedding model or reranker for a technical or analytical RAG system, it will show you weaknesses that broad benchmarks like BEIR hide, because BEIR queries can often be solved on surface overlap alone. Many strong models that look excellent on BEIR drop sharply on BRIGHT, which tells you something useful about how they will behave on hard, indirect questions.
The licence is CC BY 4.0, so you can use it commercially as long as you credit the source. The main caveat is scope: it is a focused, reasoning-first test set, not a general-purpose retrieval benchmark, so read its scores alongside broader ones like BEIR and MTEB rather than on their own. Use BRIGHT to stress-test the hard cases, and the wider benchmarks to check everyday performance.
Related sources
BEIR
A collection of 18 information retrieval datasets spanning ad-hoc web search, question answering, fact verification, and duplicate question retrieval. Built by aggregating existing datasets, some originally created for other tasks and converted to retrieval format. Tests how well retrieval models generalise to unseen domains without fine-tuning, which matters for real-world RAG.
CoIR (Code Retrieval Benchmark)
A code retrieval benchmark spanning diverse programming tasks and languages. Built for evaluating how well models retrieve relevant source code, which matters if you are building RAG over codebases.
FreshStack
A 2025 framework and benchmark for retrieval over fast-moving technical documentation. It pairs real Stack Overflow questions and answers with chunked GitHub code and docs across five niche domains, giving code and docs RAG a hard, contamination-resistant test.
LegalBench-RAG
The first open benchmark for the retrieval step of legal RAG. It offers 6,858 human-annotated query-answer pairs over a corpus of roughly 79 million characters, with character-level ground-truth spans across contracts and privacy policies, assembled from CUAD, MAUD, ContractNLI and PrivacyQA.