NLP Datasets

NLP Datasets is a single, well-known GitHub README that catalogues free and public-domain text datasets, the raw material for most retrieval and language work. Maintained by the developer niderhoff, it lists entries alphabetically, each with a short description, an approximate size where known, and a link to the source. The coverage leans towards general text corpora, conversational and dialogue data, sentiment-labelled collections, and summarisation sets.

There is nothing to download from the list itself: it is a signpost, not a dataset. You read the README on GitHub, or clone the repository if you want the Markdown locally, then follow the links out to each dataset's own home to get the data. That makes it a fast way to get oriented when you know you need text of a certain shape (say, dialogue or summarisation pairs) and want to see the well-established options in one place before committing.

For RAG, treat it as a discovery tool at the very start of a project. It is most useful when you are assembling a training or evaluation corpus and want language data specifically, rather than tabular, geospatial, or image sets. Once you have a shortlist, the real work of checking size, format, and quality happens at each dataset's source.

The caveats are the usual ones for a community list. It is curated by hand, so it can lag behind newer releases, and outbound links to older datasets sometimes rot; always confirm a dataset is still available and note its individual licence at its home before you build on it, because the list gathers sources under many different terms. As a meta-resource it complements, rather than replaces, the larger and more actively indexed catalogues: the Hugging Face Datasets Hub for searchable, ready-to-stream data, and broader directories such as Awesome Public Datasets when your needs go beyond text.