LegalBench-RAG

LegalBench-RAG, built by ZeroEntropy, is the first open benchmark aimed squarely at the retrieval step of legal RAG rather than the generation step. It provides 6,858 query-answer pairs over a corpus of roughly 79 million characters, with character-level ground-truth spans annotated by people, so you can measure not just whether a system found the right document but whether it retrieved the precise passage a lawyer would cite. The material spans non-disclosure agreements, merger and acquisition contracts, commercial agreements, and privacy policies, and the benchmark is assembled from four existing legal datasets, namely CUAD, MAUD, ContractNLI and PrivacyQA.

You access it from the GitHub repository, where the benchmark ships as JSON test cases (each a query plus an array of ground-truth snippets) alongside the underlying raw text corpus. A smaller LegalBench-RAG-mini split is included for fast iteration. Scoring focuses on precise retrieval, rewarding systems that return minimal, highly relevant spans rather than whole documents, which maps closely to how legal answers must be grounded.

For RAG and AI work this is an evaluation tool, not a knowledge base to deploy. Use it to compare embedding models, chunking strategies, and rerankers on legal text, and to catch the common failure where a retriever surfaces the right contract but the wrong clause. It is best for anyone building contract review, due diligence, or compliance assistants who needs a defensible retrieval score.

On licensing, the repository code and benchmark packaging are released under the MIT licence, but the underlying documents come from the four component datasets, each with its own terms, where CUAD is CC BY 4.0 while MAUD, ContractNLI and PrivacyQA carry separate conditions. Check every component before commercial redistribution, and preserve attribution where required. There is no territory restriction. Alongside our legal corpora such as pile-of-law and the Cambridge Law Corpus, which give you text to build on, LegalBench-RAG gives you the yardstick to prove your legal retrieval actually works, much as BEIR does for general-domain retrieval.