OSCAR

OSCAR, the Open Super-large Crawled Aggregated coRpus, takes Common Crawl and does one very useful thing with it: it runs the text through a language classifier and splits it into clean per-language slices, so instead of one giant multilingual pile you get separate corpora for each of more than 150 languages. For many of those languages, especially the ones the big English-first datasets barely touch, OSCAR is among the largest openly available sources of web text anywhere.

You get it through the OSCAR project site and Hugging Face under oscar-corpus, where it arrives as JSONL, one document per line. Some of the larger or more sensitive language sets are gated behind a quick access request, so factor in a short approval step. Later releases, such as the 2022 versions, add document-level annotations for quality and content and offer both deduplicated and original variants, so you can filter before you index or train.

It is a natural starting point for non-English and multilingual pretraining, for building a knowledge base in a particular language, or for cross-lingual RAG where retrieval needs to work across languages. If your project lives outside English, OSCAR is one of the first sources to reach for.

It is close to raw web text, so quality varies, and it varies most for low-resource languages, where you will meet language-identification errors, boilerplate, and some adult or offensive material: plan on filtering and deduplicating before use. On licensing, the metadata is CC0 while the underlying page content stays under its original terms, so although the corpus itself permits commercial use, redistributing the actual text needs the same care you would give raw Common Crawl.

It sits alongside mC4, CC-100, CulturaX, and the newer FineWeb-2 and HPLT corpora. OSCAR's edge is its long tail of languages and its per-language cleanliness rather than the heaviest filtering.