Skip to content
RAG Repo

PubChem is one of the largest open chemistry resources anywhere, run by the US National Institutes of Health. It is organised into three linked databases: Substances (contributed records, often duplicated across depositors), Compounds (unique, normalised chemical structures deduplicated from those substances), and BioAssays (the results of biological activity experiments). Most entries are small molecules, but you will also find nucleotides, carbohydrates, lipids, peptides, and modified macromolecules. For each compound you get the structure, standard identifiers such as InChI and canonical SMILES, calculated and measured physical properties, links to patents, and a large amount of health, safety, and toxicity information pulled together from many sources.

For occasional lookups the PUG REST API is the easiest way in: you can request a compound by name, identifier, or structure and get back JSON. For a retrieval system you will usually want the bulk FTP downloads, available as SDF (the standard chemical structure format), CSV, and XML. If you want to build a knowledge graph, the PubChemRDF project publishes the Compound, Substance, and BioAssay data as RDF, a way of writing facts as subject-predicate-object triples, amounting to roughly 80 billion of them, which you can load into a triple store and query with SPARQL.

For RAG, a useful pattern is to keep the structured properties in a database for exact lookups and embed the text-rich parts (the safety summaries, descriptions, and pharmacology sections) for semantic retrieval, since a language model reasons better over prose than over a raw property table. It suits chemistry assistants, drug-discovery tools, and safety or regulatory question answering.

As a work of the US Government, PubChem is in the public domain, so you can use it freely in commercial products with no licence conditions and no attribution requirement, which is unusually generous. Two caveats: the annotations are aggregated from third parties of varying reliability, so treat individual data points as leads to verify rather than gospel, and if you want manually curated, literature-backed bioactivity data at higher confidence, ChEMBL is the natural companion.

chemistrydrug-discoveryus-governmentpublic-domainstructuredbioactivity

Related sources