Open access repositories of scholarly works, including preprint servers, citation databases, and full-text paper archives across scientific disciplines.
7sources
Academic and scientific literature is where RAG earns its keep. When a system needs to answer with real evidence, citing a paper rather than a paraphrase, these repositories of research articles, preprints, and citation data are what make that possible. A preprint is a paper shared before formal peer review, which means you often get access to findings months before they appear in a journal. For any research, technical, or medical assistant, this category is close to essential.
The big trade-off is full text versus metadata. Some sources give you the complete article, ready to chunk into passages for retrieval. Others give only titles, abstracts, and citation links, which are lighter to store and still useful for discovery, but cannot ground a detailed answer on their own. Decide early which you need, because retrofitting full text later is painful. Coverage also varies by field: some corpora lean heavily towards biomedicine or computer science, so check the subject mix against your domain.
Watch the licensing closely here. “Open access” describes how you can read a paper, not always how you can reuse it. Many articles are free to view but still carry restrictive terms, and permissions can differ paper by paper within the same collection. Formats can be fiddly too: PDFs need careful extraction, and machine-readable formats like JATS XML, a publishing standard for scholarly articles, are far kinder to a pipeline.
The sources below span full-text archives, preprint servers, and citation graphs, so you can match coverage and format to your project.
An open-access preprint server for physics, mathematics, computer science, quantitative biology, statistics, and more, holding over 2.5 million papers. You can pull the full archive in bulk from Amazon S3, or harvest the metadata through OAI-PMH, a standard protocol for sharing records between repositories.
An aggregator of open-access research papers that harvests from thousands of repositories and journals worldwide. It holds over 300 million metadata records and more than 40 million full-text articles, all reachable through one search API.
A nonprofit DOI registration agency that publishes metadata for over 150 million scholarly works, including journal articles, books, and conference proceedings. A DOI is the permanent identifier assigned to each work, and Crossref shares its metadata through a free API.
A free, open catalogue of the world's scholarly works, authors, venues, institutions, and research topics. It succeeds Microsoft Academic Graph and holds hundreds of millions of works with rich metadata and citation links.
The US National Library of Medicine's database of biomedical and life sciences literature. PubMed indexes over 36 million citations and abstracts, and PubMed Central (PMC) adds free full-text access to a growing subset. You can download the data in bulk over FTP.
Allen AI's academic graph, covering hundreds of millions of papers linked by billions of citations. S2ORC, the Semantic Scholar Open Research Corpus, is the downloadable dataset with full text and parsed references, drawn from arXiv, PubMed, Crossref, publishers, and web crawlers.
A database that tracks where scholarly articles can be read legally for free, covering over 30 million open-access papers. Maintained by the nonprofit OurResearch, it links each article's DOI to full-text versions hosted across repositories and journals.