LongEmbed
LongEmbed tests retrieval over documents that are far longer than the norm, with an average length of more than 5,500 words. Most established benchmarks lean on short passages of at most 512 tokens, the small chunks of text an embedding model reads in one pass, which tells you little about how a model copes when the relevant information is buried deep in a long document.
This makes LongEmbed a useful check if your RAG system works with long reports, contracts, or articles rather than short snippets. It is released under the MIT licence.
Related sources
BEIR
A collection of 18 information retrieval datasets spanning ad-hoc web search, question answering, fact verification, and duplicate question retrieval. Built by aggregating existing datasets, some originally created for other tasks and converted to retrieval format. Tests how well retrieval models generalise to unseen domains without fine-tuning, which matters for real-world RAG.
BRIGHT
A reasoning-intensive retrieval benchmark. It requires genuine reasoning to connect queries with relevant documents rather than keyword or semantic overlap, revealing model weaknesses that BEIR misses.
CoIR (Code Retrieval Benchmark)
A code retrieval benchmark spanning diverse programming tasks and languages. Built for evaluating how well models retrieve relevant source code, which matters if you are building RAG over codebases.
MIRACL
A multilingual retrieval benchmark built from human-annotated relevance judgements over Wikipedia articles, covering 18 languages. Widely used for evaluating retrieval within a single language across a broad set of languages.