Cohere Wikipedia Multilingual Embeddings (2023-11)
This dataset takes the full Wikipedia dump from November 2023, covering more than 300 languages, and does the expensive part of a RAG pipeline for you: it splits every article into paragraph-sized passages and converts each one into an embedding (a numeric vector that captures the meaning of the text) using the Cohere Embed V3 multilingual model. There are close to 250 million passage embeddings in the collection.
Because everything is pre-embedded, you can load it straight into a vector database and start running semantic search, where results are matched by meaning rather than exact keywords. The multilingual model shares one vector space across languages, so a query written in any language can return relevant passages written in others.
The embeddings are released under Apache 2.0, while the underlying Wikipedia text remains under CC BY-SA. Note that whether embeddings derived from ShareAlike text count as a derivative work is genuinely unsettled, so factor that ambiguity into any commercial plans.
Related sources
Cohere Wikipedia 22-12 Embeddings (per-language)
Wikipedia encoded with the Cohere multilingual-22-12 embedding model, with embeddings (numeric vectors that capture meaning) computed on each article's title plus text. Published per language, covering many including Arabic, German, English, Spanish, French, Hindi, Italian, Japanese, Korean, Simple English, and Chinese. You can stream it rather than download it in full, which matters given the size.
LAION-400M with CLIP Embeddings
400 million image-text pairs filtered from Common Crawl using CLIP, a model that scores how well an image matches a caption, and released alongside their pre-computed CLIP embeddings and kNN indices for fast similarity search. The ready-made embeddings make it unusually easy to work with. Note that it ships image URLs, not the images themselves, so some links have decayed.