Project Vaani

Project Vaani is a large corpus of Indian speech built by the Indian Institute of Science (IISc) in Bangalore together with ARTPARK, and funded by Google, with the aim of capturing the spoken language landscape across every district of India. Speakers are recorded describing images shown to them, which yields spontaneous, conversational audio rather than scripted reading. The open-sourced version listed here covers roughly 31,278 hours of audio from about 156,000 speakers across 105 languages and dialects in 165 districts and 31 states and union territories, of which around 2,122 hours carry transcriptions, often in the local script. The eventual target is more than 150,000 hours spanning all 773 districts, so the published totals will keep growing.

You get the data from the Hugging Face Hub, where it is published by the ARTPARK-IISc organisation and stored as Parquet covering audio, image and text, loadable per language through the datasets library with optional streaming for the larger subsets. Access is gated: you accept the dataset conditions and sign in with a Hugging Face account before downloading, which is why we list it as limited availability rather than fully open.

For RAG and AI work, Vaani is training and evaluation material for the speech-to-text stage that turns Indian-language audio into the text you later chunk, embed and retrieve, and it is especially valuable for low-resource languages and dialects that mainstream recognition handles poorly. It is best for building or benchmarking multilingual Indian speech recognition and for fine-tuning on spontaneous, accented speech.

The headline licence is CC BY 4.0, so commercial use is permitted with attribution and there is no share-alike obligation. The caveat matters: some subsets mirrored on the government AIKosh portal, such as the Kanniyakumari Tamil dialect transcripts, are listed under CC BY-NC 4.0, which forbids commercial use, so check the licence of each subset before building anything commercial on it. If you list Multilingual LibriSpeech, treat Vaani as its Indian-language, spontaneous-speech counterpart.