AMPS

AMPS (Auxiliary Mathematics Problems and Solutions) was released by Dan Hendrycks and collaborators as the pretraining companion to the MATH benchmark. It pairs two very different kinds of content. The first is more than 100,000 problems drawn from Khan Academy, each with a worked, step-by-step solution written in LaTeX, the typesetting language maths is normally published in. The second is over 5 million problems generated programmatically with Mathematica scripts, spanning topics from algebra and geometry to calculus and number theory. Together they come to roughly 23 GB.

You get it from the MATH repository on GitHub (hendrycks/math), where AMPS ships as a downloadable archive of files organised by subject and problem type. There is no API or streaming layer: you clone or download, unpack, and parse the LaTeX and JSON yourself. A practical tip is to keep the LaTeX intact when you chunk, because the notation is the content; stripping it to plain text destroys the maths. Render or tokenise it in a way your embedding model can handle rather than flattening equations into plain ASCII.

AMPS is best when your RAG system needs to retrieve worked examples rather than bare answers: a tutoring assistant that shows its reasoning, a step-by-step solver, or a corpus for grounding a maths model's explanations. The human-written Khan Academy portion gives natural, teaching-style prose, while the Mathematica portion supplies scale and breadth of problem types.

The main caveat is uneven quality between the two halves. Machine-generated problems are consistent but can feel templated and repetitive, and some carry little explanatory text, so a relevance or quality filter helps before indexing. On licensing you are in good shape: AMPS is MIT-licensed, which permits commercial use with no share-alike obligation, so any derivative retrieval index is yours to ship. Alongside quality-scored corpora like AutoMathText or the larger MegaMath, AMPS stands out for its clean, explicitly worked solutions rather than sheer token count.