FinePDFs

FinePDFs is HuggingFace's large-scale corpus built entirely from PDF documents, released by the same FineWeb team behind FineWeb and FineWeb2. It gathers roughly 3 trillion tokens from about 475 million documents across 1,733 languages, with English accounting for more than 1.1 trillion tokens. The data was drawn from 105 Common Crawl snapshots spanning mid-2013 to February 2025, with some documents refetched from the live web, then processed through HuggingFace's open datatrove pipeline for text extraction, language identification, deduplication and anonymisation of personal information.

The point of a PDF-only corpus is coverage that HTML web crawls miss. PDFs hold dense, long-form material: government and corporate reports, technical manuals, academic papers, legal filings and reference documents that rarely appear as clean web pages. Documents here are unusually long, which suits retrieval tasks that need sustained context rather than short article snippets.

Access is through HuggingFace. The data is organised by language and script (for example eng_Latn, cmn_Hani, arb_Arab) and streams with the datasets library, so you can pull a single configuration and begin chunking without downloading the whole set. For RAG, treat it as a filtered text pool that still needs your own chunking and embedding. It is a strong base for multilingual knowledge bases and for fine-tuning on long, formal documents.

On licensing, the compilation is released under ODC-BY 1.0, which permits commercial use provided you attribute the dataset. The important caveat is that the underlying PDF content keeps its original copyright and each source's terms, since much of it comes from Common Crawl rather than cleared public-domain material. Treat the text as filtered web-sourced content, not rights-cleared data. Next to FineWeb and FineWeb2, which rebuild HTML web pages, FinePDFs is the PDF counterpart: complementary long-form coverage from the same team and the same open tooling.