Pre-embedded & RAG-ready

Datasets that come already split into passages and converted into embeddings, so you can plug them straight into a vector database and skip the most expensive part of building a pipeline.

3sources

Pre-embedded & RAG-readyOpen

Cohere Wikipedia 22-12 Embeddings (per-language)

Wikipedia encoded with the Cohere multilingual-22-12 embedding model, with embeddings (numeric vectors that capture meaning) computed on each article's title plus text. Published per language, covering many including Arabic, German, English, Spanish, French, Hindi, Italian, Japanese, Korean, Simple English, and Chinese. You can stream it rather than download it in full, which matters given the size.

wikipediaembeddingsmultilingual

Pre-embedded & RAG-readyOpen

Cohere Wikipedia Multilingual Embeddings (2023-11)

The November 2023 Wikipedia dump across 300+ languages, split into passages and turned into embeddings (numeric vectors that capture meaning) with the Cohere Embed V3 multilingual model. Close to 250 million paragraph embeddings in total. Drop it into a vector database for semantic search over all of Wikipedia, or use it as a knowledge source for multilingual RAG, including cross-lingual search where a query in one language returns relevant results from another.

wikipediaembeddingsmultilingual

Pre-embedded & RAG-readyOpen

LAION-400M with CLIP Embeddings

400 million image-text pairs filtered from Common Crawl using CLIP, a model that scores how well an image matches a caption, and released alongside their pre-computed CLIP embeddings and kNN indices for fast similarity search. The ready-made embeddings make it unusually easy to work with. Note that it ships image URLs, not the images themselves, so some links have decayed.

image-textclipembeddings

Cohere Wikipedia 22-12 Embeddings (per-language)

Cohere Wikipedia Multilingual Embeddings (2023-11)

LAION-400M with CLIP Embeddings

Other categories