Cohere Wikipedia 22-12 Embeddings (per-language)
This dataset encodes Wikipedia using the Cohere multilingual-22-12 embedding model, converting each passage into an embedding, a numeric vector that captures its meaning, computed on the article's title plus its text. It is split by language, so you can pull down just the languages you need rather than the whole collection.
Coverage spans many languages, including Arabic, German, English, Spanish, French, Hindi, Italian, Japanese, Korean, Simple English, and Chinese. Because the passages are already embedded, you can load a language into a vector database and run semantic search, matching results by meaning rather than exact keywords, without building an embedding pipeline yourself.
The files can be streamed instead of downloaded in full, which is helpful given the size. The embeddings are released under Apache 2.0, while the underlying Wikipedia text stays under CC BY-SA, and whether embeddings of ShareAlike text count as a derivative work remains unsettled.
Related sources
Cohere Wikipedia Multilingual Embeddings (2023-11)
The November 2023 Wikipedia dump across 300+ languages, split into passages and turned into embeddings (numeric vectors that capture meaning) with the Cohere Embed V3 multilingual model. Close to 250 million paragraph embeddings in total. Drop it into a vector database for semantic search over all of Wikipedia, or use it as a knowledge source for multilingual RAG, including cross-lingual search where a query in one language returns relevant results from another.
LAION-400M with CLIP Embeddings
400 million image-text pairs filtered from Common Crawl using CLIP, a model that scores how well an image matches a caption, and released alongside their pre-computed CLIP embeddings and kNN indices for fast similarity search. The ready-made embeddings make it unusually easy to work with. Note that it ships image URLs, not the images themselves, so some links have decayed.