Open datasets for fine-tuning LLMs
Fine-tuning needs example data, not a web corpus. A practical tour of open instruction, reasoning, and tool-use datasets for fine-tuning LLMs, with the commercial-licence catch that trips people up.
Blog
Notes on RAG, data repositories, and the messy business of data licensing.
Meet the RAG Repo API: the directory as open JSON
Every source in the directory is now published as a free, read-only JSON API, with a licence-aware catalogue and a what's-new feed. Here is what it exposes and how to build on it.
Building a structural-biology RAG system: the data you actually need
The Protein Data Bank, AlphaFold, and UniProt are extraordinary open resources, but they are 3D structures and sequences, not prose. How to actually plug structural-biology data into a RAG or AI system, and the literature layer that makes it useful.
Read Building a structural-biology RAG system: the data you actually need
The open web is closing: what it means for your training data
A wave of settlements, regulation and access controls is reshaping where AI training and RAG data can come from. What changed, why it matters, and why copyright-clean corpora are suddenly the interesting ones.
Read The open web is closing: what it means for your training data
The best open datasets for RAG in 2026
A curated tour of the open datasets we reach for most when building RAG and AI systems, from web-scale corpora and encyclopaedic knowledge to legal text and evaluation benchmarks.
Common Crawl vs FineWeb vs C4 vs RedPajama: choosing a web corpus
The big open web corpora look alike from the outside. Here is how Common Crawl, FineWeb, C4, RedPajama and their cousins actually differ in filtering, size, cleanliness and licence, and how to choose between them.
Read Common Crawl vs FineWeb vs C4 vs RedPajama: choosing a web corpus
How to choose a dataset licence for a commercial RAG product
A practical guide to reading dataset licences before you ship: permissive versus share-alike versus non-commercial, attribution, and the trap where the content licence differs from the dataset licence.
Read How to choose a dataset licence for a commercial RAG product
Hardware for a local RAG system: what you actually need
A practical guide to sizing hardware for local RAG, from RAM and GPU for embeddings and local models to the storage your corpus and vector index will really consume.
Read Hardware for a local RAG system: what you actually need
Building a legal RAG system: the data sources you actually need
Case law, legislation and regulatory filings for legal RAG across the UK, US and EU, with notes on licensing and how to structure legal text for retrieval.
Read Building a legal RAG system: the data sources you actually need
Evaluating your RAG system: the benchmarks and datasets that matter
You cannot improve what you cannot measure. The retrieval and RAG evaluation datasets worth knowing, from BEIR and MS MARCO to MTEB and purpose-built RAG benchmarks, plus how to build your own eval set.
Read Evaluating your RAG system: the benchmarks and datasets that matter
Chunking strategies for RAG retrieval
How you split documents shapes retrieval quality more than almost anything else. A practical tour of fixed-size, recursive, structure-aware, semantic and contextual chunking, and how to choose.
Multilingual RAG: where to get non-English data
Open multilingual text and speech corpora for building RAG beyond English, from CulturaX and FineWeb2 to regional and community-driven datasets, with notes on coverage, quality and licences.
Wikipedia and Wikidata for RAG: the practical guide
The two most-used knowledge sources in RAG, done properly: which dumps to use, how the text and the knowledge graph complement each other, licensing, and pre-embedded options.
The licensing trap in RAG data
Why data licences catch RAG teams out, and how to spot share-alike, non-commercial, and not-quite-open terms before they become a problem.
How to choose your first RAG dataset
Practical, friendly advice for picking your first RAG dataset: start small, check the licence and format, and match the data to what you are building.
Welcome to RAG Repo
RAG Repo is a free, curated directory of open datasets and knowledge bases for building RAG systems, plus plain-English guides to help you use them.
Prefer a feed? Subscribe via RSS.