Cohere Wikipedia 22-12 Embeddings (per-language)
This dataset encodes a December 2022 snapshot of Wikipedia using the Cohere multilingual-22-12 embedding model, converting each passage into an embedding, a numeric vector that captures its meaning, computed on the article's title plus its text. It ships one dataset per language, so you can pull down just the languages you need rather than the whole collection. Coverage spans many languages, including Arabic, German, English, Spanish, French, Hindi, Italian, Japanese, Korean, Simple English, and Chinese.
The files live on HuggingFace. Because the collections are large, the sensible way in is to stream them (`streaming=True` on the HuggingFace `datasets` loader) rather than download a language in full, then write the passages and their vectors straight into a vector database. The big win is that the passages are already embedded, so you skip building and running an embedding pipeline and can stand up multilingual semantic search, matching by meaning rather than exact keywords, in an afternoon.
There is one thing you must get right. These vectors were produced by a specific Cohere model, so at query time you have to embed the user's question with the same multilingual-22-12 model (via Cohere's API) for the numbers to line up. Mix in a different embedding model and the search results become meaningless. Keep that dependency in mind before committing, and remember the text is a 2022 snapshot, so anything newer than that will be missing.
On licensing, the embeddings are released under Apache 2.0 while the underlying Wikipedia text stays under CC BY-SA, and whether embeddings of share-alike text count as a derivative work remains unsettled, so tread carefully if you plan to redistribute.
If you want wider coverage, Cohere's newer Wikipedia 2023-11 Embed V3 dataset spans 300+ languages and a fresher snapshot; this per-language 22-12 release is lighter to start with when you only need a handful of languages.
Related sources
Cohere Wikipedia Multilingual Embeddings (2023-11)
The November 2023 Wikipedia dump across 300+ languages, split into passages and turned into embeddings (numeric vectors that capture meaning) with the Cohere Embed V3 multilingual model. Close to 250 million paragraph embeddings in total. Drop it into a vector database for semantic search over all of Wikipedia, or use it as a knowledge source for multilingual RAG, including cross-lingual search where a query in one language returns relevant results from another.
Google Satellite Embedding (AlphaEarth Foundations)
Global annual satellite embeddings produced by Google and Google DeepMind's AlphaEarth Foundations model. Every 10 metre pixel holds a 64 dimensional vector summarising a year of multi-sensor Earth observation, available yearly from 2017.
LAION-400M with CLIP Embeddings
400 million image-text pairs filtered from Common Crawl using CLIP, a model that scores how well an image matches a caption, and released alongside their pre-computed CLIP embeddings and kNN indices for fast similarity search. The ready-made embeddings make it unusually easy to work with. Note that it ships image URLs, not the images themselves, so some links have decayed.
Major TOM
An open, globally dense collection of embeddings and ML-ready imagery derived from Copernicus Sentinel data, published by ESA Phi-lab. The embedding expansions run to well over 100 billion vectors covering most of the Earth.