Datasets pairing images with text, for RAG systems that retrieve across pictures and words or need visual grounding.
7sources
A cleaned image alt-text dataset for automatic image captioning, produced by Google Research. Alt-text pulled from the web is filtered and generalised (hypernymed), replacing specific names with broader categories. Available in 3.3M (CC3M) and 12M (CC12M) variants.
A benchmark and dataset collection for training multimodal models. It provides a shared pool of image-text candidates and an evaluation framework so teams can test different data-curation strategies against a common yardstick.
An image-text dataset of over 5.8B examples, built by filtering Common Crawl with a CLIP model that scores how well an image matches a caption. Includes 2.32B English pairs, 2.26B multilingual pairs, and 1.27B not tied to any particular language. Provides URLs, not the images themselves.
A web-scale dataset of 141M multimodal English web documents that interleave text and images, containing 353M images and 115 billion tokens. Unlike image-caption pair datasets, it keeps whole documents with text and images in their original order. Used to train the IDEFICS models.
An image-text dataset built only from materials marked with a Public Domain Mark or released under Creative Commons Zero (CC0). 12.4M image-caption pairs, with a 3.3M subset called PD3M, sized to match the Conceptual Captions datasets while staying copyright-clean for commercial use.
A Wikipedia-based image-text dataset for multimodal, multilingual machine learning. It pairs images with their Wikipedia captions and the surrounding article text across many languages, giving richer context than a single caption.
A multimedia research dataset of 100M Flickr photos and videos with their metadata, all published under Creative Commons licences. The exact licence varies from one item to the next, so check each one before you reuse it.