Speech & audio

Speech and audio datasets are collections of recorded spoken language, usually paired with transcripts. They matter for RAG, where a language model answers from retrieved passages of text, because so much human knowledge starts as speech: lectures, meetings, podcasts, support calls. Turn that audio into text with automatic speech recognition, the technology that transcribes spoken words, and it becomes searchable content your system can retrieve like any other document. These datasets are also what you use to train or evaluate the transcription step itself.

The first thing to weigh is the kind of speech. Read speech, where someone narrates written text, as in LibriSpeech or LJ Speech, is clean and consistent, which is great for training a reliable transcriber but unlike real conversation. Spontaneous, crowd-recorded speech like Mozilla Common Voice or The People’s Speech is messier and far closer to how people actually talk. After that, look at total hours, how many different speakers and accents are represented, and whether the transcripts are human-checked or automatically generated.

Licensing and privacy deserve real attention here, more than with plain text. Terms vary widely: Common Voice is released under CC0, meaning effectively public domain, while other sets carry research-only or non-commercial restrictions. Because these are recordings of real people’s voices, check that the speakers consented to the use you have in mind. Audio quality, background noise, and how tightly the transcript aligns to the audio all shape how usable a set really is.

The datasets below span clean read speech, diverse crowd-sourced voices, and speaker collections for voice-focused work.