Library of Congress Selected Digitized Books
The Selected Digitized Books data package is a collection of optical character recognition (OCR) full text drawn from 90,414 digitised books held by the Library of Congress, selected from over 118,000 digitised titles as of August 2022. It is published and maintained by LC Labs, the Library's experimental data team, and most of the works were published in the United States before the 1930s, in English, with a smaller number of foreign language titles. Because the books predate current United States copyright, they sit in the public domain and the Library states they are free to use and reuse.
Access is by direct download rather than a gated service. The package is hosted on the LC Labs data site, where you can pull the full text as .txt and .json files, metadata as CSV, JSON and JSON Lines, and manifests that list every file with its identifier, size, MD5 hash and URL. A 1,000 item sample is offered for quick inspection, and zipped bulk downloads cover all or part of the set. The Library also provides Python notebooks on GitHub for scripted retrieval, and the underlying loc.gov JSON and YAML API can be queried directly.
For RAG and AI work this is a clean, provenance clear book corpus you can chunk and embed without the licensing doubts that attach to web crawls or scraped book collections. It suits retrieval over historical and public domain material, long form reading passages, and pretraining or fine tuning where copyright hygiene matters. The main caveats are practical rather than legal, in that the text is raw OCR, so expect recognition errors on older typefaces, and the descriptive terminology in some historical works reflects the attitudes of their time. The snapshot dates from 2022, so newer digitised titles will not be present. It also feeds into the Common Pile corpus we list, so if you want this material already blended with other openly licensed sources, that is the natural neighbour here.
Related sources
Google Books Ngrams
Word and phrase frequency data drawn from Google's digitised book collection, spanning centuries of published text. It counts how often words and short phrases appear by year, which is useful for historical language analysis and time-aware RAG features.
Institutional Books 1.0
A 242 billion token dataset of roughly 983,000 public-domain volumes digitised from Harvard Library's collections, spanning more than 250 languages, with both raw and post-processed OCR text plus rich bibliographic metadata.
LibriVox
A volunteer project providing free public domain audiobooks. Volunteers record themselves reading works whose copyright has expired, which makes it a handy source of speech data for training speech-to-text models and building multimodal RAG.
Open Library
An Internet Archive project building a web page for every book ever published. It holds more than 20 million catalogue records and lends many titles digitally, making it a rich source of book metadata for RAG systems.