HPLT (High Performance Language Technologies)
HPLT (High Performance Language Technologies) is an EU-funded project focused on producing open datasets and models for many languages. Its monolingual text release, MonoHPLT, covers 75 languages and is built from a mix of web crawls and the Internet Archive, giving you an alternative source of multilingual text that does not rely solely on Common Crawl.
Where possible the data is released under CC0, effectively placing it in the public domain with no rights reserved, so commercial use is permitted without attribution. The project also publishes its processing tools and documentation, which helps if you want to understand how the corpus was cleaned before you build on it.
Related sources
AI4Bharat (IndicCorp)
A collection of corpora, models, and benchmarks for Indian languages, produced by IIT Madras. It covers major Indic languages with monolingual corpora, parallel translation data, and evaluation sets.
CC-100
A high-quality multilingual corpus derived from Common Crawl. It splits every document into separate paragraphs, so it is effectively a paragraph-level rather than document-level corpus, which matters for your chunking strategy.
CulturaX
A multilingual text dataset of 6.3 T tokens across 167 languages, built for training large language models. It combines cleaned versions of mC4 and OSCAR, two large web corpora, after a multi-stage filtering pipeline.
GlotCC
An open, broad-coverage corpus and processing pipeline built from Common Crawl, targeting minority and low-resource languages that larger corpora tend to miss. It fills gaps left by datasets focused on higher-resource languages.