Nemotron-CC-v2

Nemotron-CC-v2 is NVIDIA's second-generation English web corpus for large language model pretraining, built on top of the original Nemotron-CC pipeline. It reprocesses Common Crawl with eight additional snapshots spanning 2024 to 2025, applies global deduplication and English filtering, and preserves the mathematics and code passages that generic web filters often discard. On top of the organic crawl it layers synthetically rephrased text, generated with Qwen3-30B-A3B, and a large set of synthetic diverse question and answer pairs that are also translated into fifteen languages, giving the corpus a multilingual and instruction-flavoured character rather than plain scraped prose.

The dataset card reports roughly 6.6 trillion tokens across its subsets, with a total download of about 10.3 TB, all published as Parquet. Access is through the Hugging Face dataset page, where you download bulk Parquet shards after accepting NVIDIA's data agreement. There is no query API: this is a pretraining and fine-tuning corpus meant to be streamed or sharded at scale, not a retrieval index you point a RAG app at directly.

For RAG work it is most useful upstream: as training or continued-pretraining data for a base or embedding model, or as a raw pool you clean, chunk, and embed yourself. The mathematics, code, and multilingual question and answer slices make it a reasonable starting point for domain or language coverage.

Watch the licensing. This is not an open dataset: the NVIDIA Data Agreement for Model Training permits internal model training but prohibits redistributing the data or placing it under an open-source licence, and the underlying web content and synthetic generations carry their own upstream terms. Treat the crawled text as subject to original site rights. If you want a more permissively licensed web corpus, look at FineWeb or the original Common Crawl archives, both likely already in this directory.