FineWeb-Edu

FineWeb-Edu is the educational slice of HuggingFace's FineWeb corpus, which is itself a cleaned and deduplicated rebuild of Common Crawl snapshots. To make it, HuggingFace trained a classifier on roughly 450,000 annotations generated by Llama3-70B-Instruct to score each page for educational quality, then kept only the higher-scoring documents. The default release filters out pages scoring below 3, discarding about 92 percent of FineWeb and leaving around 1.3 trillion tokens. A companion release, fineweb-edu-score-2, uses a looser threshold of 2 and preserves about 5.4 trillion tokens while still outscoring the full FineWeb on downstream benchmarks.

You access it on HuggingFace as Parquet files that stream directly with the datasets library, so you can pull a single shard and start chunking without downloading the whole set. Sample subsets (10BT, 100BT and 350BT) are provided for smaller experiments, and each row carries its educational score so you can filter further. The filtering pipeline (built on HuggingFace's datatrove) and the classifier are open, letting you trace why a document survived or rerun with your own thresholds.

For RAG and AI work, reach for FineWeb-Edu when passage quality matters more than sheer breadth: study assistants, general-knowledge question answering and explanatory retrieval, or as a fine-tuning and pretraining base weighted towards textbook-style prose. Compared with our full FineWeb entry it is smaller but far denser in teachable content; for non-English work see FineWeb2 and FineWeb2-HQ.

The compilation is licensed ODC-By 1.0, which permits commercial use with attribution to the dataset. The underlying web pages keep their own terms and remain subject to Common Crawl's terms of use, so treat the text as filtered web content rather than material cleared for any purpose. There is no territory exclusion.