MIRIAD

MIRIAD (Medical Instruction and RetrIeval Dataset) is a large collection of medical query-response pairs distilled from peer-reviewed literature by a team from ETH Zurich and Stanford. It contains 5,821,948 QA pairs, each rephrased from and grounded in a sentence-based passage drawn from around 894,000 medical articles filtered from the S2ORC corpus. The pairs span 56 medical topics and disciplines, and were produced by a semi-automated pipeline combining LLM generation, rule-based filtering, grounding, and human annotation.

The data is hosted on Hugging Face in Parquet and loads directly with the datasets library, for example load_dataset("miriad/miriad-5.8M"). Two versions are published: the full 5.8M release, and a smaller 4.4M subset that has passed additional quality-control steps. Because every answer is tied back to a source passage, the dataset suits both bulk download for fine-tuning and use as a retrieval corpus.

For RAG and AI work it is best used as a ready-made medical knowledge base: the grounded QA structure gives you retrievable, citation-linked passages, and the authors report accuracy gains over unstructured RAG baselines and improved hallucination detection when models are augmented with it. It also works as an instruction-tuning set for medical assistants.

Watch-outs matter here. The QA pairs are LLM-generated from literature rather than clinically validated, and the maintainers state the data is for academic research and educational use only, with no regulatory clearance for diagnosis or decisions about real individuals. The dataset licence is ODC-By 1.0 (attribution required), but the underlying source articles and the terms of the models used to generate the pairs carry their own conditions, so treat commercial reuse cautiously and verify at source. If you need alternatives already in this directory, consider PubMed for primary biomedical literature or S2ORC for full-text academic papers.