MathTutorBench

MathTutorBench is a benchmark for measuring the pedagogical ability of large language model tutors, built by researchers at ETH Zurich and collaborators and presented as an oral paper at EMNLP 2025. Rather than checking whether a model reaches the right answer, it scores how well a model teaches. It groups seven tasks (plus two harder variants) under three headings: math expertise (problem solving, Socratic questioning), student understanding (solution correctness, mistake location, mistake correction) and pedagogy (scaffolding generation, pedagogical instruction following). To grade open-ended teacher turns it ships a trained reward model that learns to tell expert from novice responses.

The benchmark is assembled from existing middle-school maths tutoring data: GSM8k problems, MathDial (around 2,900 tutoring dialogues with simulated students) and Bridge (700 snippets of real online tutoring by novice teachers, each revised by an expert), with MathDial and Bridge combined into a set the authors call MathDialBridge. You access it through the GitHub repository (code plus data), and the reward-model training data is published separately on HuggingFace.

For RAG this is an evaluation harness rather than a knowledge base: use it to test whether a retrieval-grounded tutor gives pedagogically sound help, locates a student's mistake and scaffolds the next step instead of blurting the solution. For an AI tutor it is directly useful for measuring tutoring behaviour and mastery-style diagnosis, complementing the content and practice sources we list (OpenStax, Khan Academy, CK-12, Siyavula) and the maths problem and proof corpora (AMPS, StackMathQA, MegaMath, OpenThoughts3, NaturalProofs): those supply material to teach with, this measures the teaching.

On licensing, take care. The benchmark carries a CC BY-SA 4.0 licence (attribution plus share-alike on derivatives), but it bundles components under their own terms, and the Bridge portion is CC BY-NC 4.0, which forbids commercial use. Anyone building a commercial tutor should treat the Bridge-derived data as off-limits and rely on the freely reusable parts, or license Bridge separately.