APEX-Agents

APEX-Agents is a benchmark from Mercor for evaluating AI agents on long, realistic professional work. It contains 480 tasks drawn evenly from three expensive knowledge professions, with 160 each in investment banking, management consulting and corporate law. The tasks were written by practising analysts, consultants and corporate lawyers, who also built the data-rich worlds (project scenarios with real documents and files) that each task runs inside. The authors estimate an experienced professional would need roughly one to two hours per task, and each task ships with a detailed rubric and a gold-standard deliverable so that outputs can be graded consistently. Mercor releases the full set, including prompts, rubrics, gold outputs, source files and metadata, alongside Archipelago, its open-source infrastructure for running and scoring agents across multiple applications.

For RAG and agent work the value is twofold. The grounding documents inside each world are a clean, professionally produced corpus you can use to study retrieval over dense enterprise material such as filings, memos and contracts. The rubrics and gold outputs make it a hard yardstick for end-to-end agent quality rather than single-turn question answering: at launch the strongest model, Gemini 3 Flash, reached only 24.0 per cent Pass@1, with GPT-5.2 close behind and open-source models under 5 per cent, so it is best treated as a stretch evaluation rather than a leaderboard you expect to saturate soon.

The dataset is published under CC BY 4.0, so attribution is required, but be careful. The Hugging Face release is gated (you must agree to share contact details first), and the accompanying terms state it is intended for model evaluation only and forbid training, fine-tuning or scraping, which sits awkwardly on top of a nominally permissive licence. Treat commercial use as restricted and check the current terms before relying on it. It complements the other RAG-specific evaluation sets we list, but targets agentic, multi-step professional tasks rather than short retrieval question answering.