AutoMathText

AutoMathText assembles roughly 200 GB of mathematical text from three complementary sources: general websites, arXiv papers, and GitHub code, building on the earlier OpenWebMath, RedPajama, and AlgebraicStack collections. The result spans everything from informal explanations and forum discussions to formal proofs and computational maths, giving you both the prose and the notation a maths-aware system needs.

What sets it apart is the scoring. Every document carries a value between 0 and 1 for its relevance, quality, and educational value, assigned automatically by the Qwen-72B language model rather than by hand. That single number is the reason to reach for this dataset: instead of building your own filtering pipeline, you can set a threshold and keep only the highest-scoring material, or lower it to cast a wider net when coverage matters more than polish.

Access is straightforward. It is published on Hugging Face in Parquet, the columnar format the datasets library reads natively, so you can stream it, filter by score on the fly, and pull just the slice you want without downloading all 200 GB. A practical tip is to experiment with the score cutoff on a sample first, since the right threshold depends on whether you are grounding a tutor, a research assistant, or a code-and-maths tool.

It suits RAG projects that need clean, quality-controlled maths content: retrieval for maths tutoring, technical question answering, or as a curated ingredient in a larger corpus. The scoring lets you trade coverage against quality deliberately rather than by guesswork.

The licence is the main thing to watch. AutoMathText is released under CC BY-SA 4.0, which permits commercial use but is copyleft: you must attribute the source and, crucially, share any derivative database you distribute under the same share-alike licence. That is fine for open work but can complicate a closed product built on redistributed text. Alongside AMPS, which offers explicitly worked solutions, and MegaMath, which offers far more raw tokens, AutoMathText's edge is its per-document quality signal.