Skip to content
RAG Repo

Stack Exchange Data Dump

Stack Exchange publishes a full export of its entire network as `XML` files, hosted on the Internet Archive and refreshed roughly quarterly. It covers every site, from Stack Overflow to smaller communities on maths, cooking, English usage, law, and more than 170 others, so you can take just the domains you need rather than the whole 80 GB-plus archive. Each site's export is split into separate files for posts, comments, votes, tags, badges, and users.

The structure is what makes this dataset special for RAG. Every question links to its answers, each answer carries a score and a flag for whether the asker accepted it, and posts have tags. That gives you quality signals for free: you can rank answers by community votes and prefer accepted answers when selecting passages to retrieve, which is a far stronger relevance cue than raw text alone provides.

There are two natural ways to build with it. Pair each question with its accepted or top-voted answer to make clean query-and-response pairs, which is ideal for evaluation sets or few-shot examples. Or index whole threads, comments included, as passages when you want the surrounding discussion and caveats. The body text is HTML, so you will strip or convert the markup, and code blocks are worth preserving carefully for developer use cases.

On licensing, everything is Creative Commons Attribution-ShareAlike. Historically different site content sat under different CC BY-SA versions, but the practical obligations are the same: commercial use is allowed, but you must attribute the authors and Stack Exchange, and share any derived text under the same share-alike terms. That share-alike condition is the main thing to weigh before you bake the content into a product.

For technical and developer question answering this is the best free source there is. Pair it with the code datasets and documentation elsewhere in this category when you need the underlying source alongside the human explanations.

question-answeringstack-overflowcommunitytechnicalcc-by-sa

Related sources