DataComp

DataComp turns the usual model benchmark on its head. Instead of fixing the data and comparing models, it fixes the model architecture and the training budget, then asks which slice of data produces the best results. In other words, the dataset you assemble is the thing under test, not the model. It grew out of the effort to understand why some image-text collections train far better vision-language models than others.

Practically, it ships a common candidate pool of image-text pairs sourced from Common Crawl at several fixed scales (from small runs up to billions of pairs), along with a standardised training and evaluation harness. You take that pool, apply your own curation idea (quality filtering, deduplication, reweighting, language or topic selection), train under the fixed recipe, and measure the effect across a suite of downstream tasks. Because everyone works from the same pool and recipe, results are directly comparable. Note that the pool is distributed as URLs plus metadata rather than the images themselves, so you download the pictures yourself, and expect some link rot over time.

For RAG (Retrieval-Augmented Generation), DataComp is not a knowledge base you retrieve answers from. Its value is upstream: it is where you learn how to curate multimodal training data, and it is a strong source of image-text pairs for training or fine-tuning the CLIP-style embedding models that power multimodal retrieval. If you are building search over images and text, the curation lessons here transfer directly to how you filter your own retrieval corpus.

The framework and annotations are CC BY 4.0, so commercial use is fine with attribution, but the underlying images carry their own varied rights and you are fetching them from the open web, so treat downstream image reuse with the same care you would any web-scraped media. It sits alongside LAION and Conceptual Captions as multimodal training data, with the distinction that DataComp is as much a methodology and benchmark as it is a dataset.