The original RAG paper (Lewis et al., 2020), explained

The term “Retrieval-Augmented Generation” comes from one paper: “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, by Patrick Lewis and eleven colleagues, posted to arXiv in May 2020 and published at NeurIPS 2020. Almost everything the field now calls RAG traces back to it, yet what the paper actually proposed is narrower and more specific than the catch-all the name has become. This is a plain explainer of where RAG came from, checked against the paper itself. If you want the concept from first principles rather than its history, start with What is RAG? instead.

Who wrote it, and when

The paper was written by Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin and co-authors at Facebook AI Research and University College London, and it appeared at NeurIPS 2020. The arXiv record is arXiv:2005.11401. Several of the same authors had just published Dense Passage Retrieval, the retriever this work builds on, which is why the two are so often cited together.

The problem it set out to solve

Large pre-trained language models, the paper observes, store a surprising amount of factual knowledge in their weights, but have a “limited ability to access and precisely manipulate” it. Two shortcomings follow. A model cannot easily tell you where an answer came from, so there is no provenance. And its knowledge is fixed at training time, so updating what it knows means retraining. For knowledge-intensive tasks, questions whose answers depend on specific facts, that is a serious limitation, and it is the gap the paper set out to close.

The architecture: parametric plus non-parametric memory

The paper’s framing is the part worth keeping. It describes a model with two kinds of memory. The parametric memory is a pre-trained sequence-to-sequence model (BART in the paper), the knowledge baked into weights. The non-parametric memory is “a dense vector index of Wikipedia, accessed with a pre-trained neural retriever”, the dense passage retriever searching an embedded Wikipedia.

The two work together in a retrieve-then-generate loop. For a given input, the retriever finds the most relevant Wikipedia passages; those passages are handed to the generator alongside the input; and the generator produces the answer conditioned on them. Crucially, the retriever and generator were trained together, end to end, with only the retrieval target left latent. This is the move that defined RAG: pairing a generator with an external, swappable index instead of relying on weights alone.

RAG-Sequence and RAG-Token

The paper defines two formulations, and the distinction is still useful vocabulary. RAG-Sequence conditions on the same retrieved passages across the whole generated answer: one set of documents is chosen, and the entire output is produced from them. RAG-Token can use different passages for different tokens, letting the model draw on one document for one part of a sentence and another document for the next. RAG-Token is more flexible for answers that stitch together several facts; RAG-Sequence is simpler and often sufficient.

What it showed

Evaluated on knowledge-intensive tasks, the models set state-of-the-art results on open-domain question answering (the kind of benchmark where a system must answer a factual question with no passage provided, such as the ones built on Natural Questions and the passage-ranking data in MS MARCO). Beyond the scores, the paper reported that RAG models “generate more specific, diverse and factual language” than a comparable parametric-only model. The two structural wins are the ones that still matter most: because the knowledge lives in an external index, you get provenance (you can point at the passage used) and updatability (you can change what the model knows by changing the index, with no retraining). We use that Wikipedia-as-knowledge-base idea constantly; see Wikipedia and Wikidata for RAG.

How today’s RAG differs

If you have built a RAG system recently, some of this will sound unfamiliar, because the common practice has drifted a long way from the 2020 design.

No joint training. The original trained retriever and generator together. Almost nobody does that now. Today’s typical RAG is training-free: an off-the-shelf embedding model and vector database for retrieval, and a large instruction-tuned LLM that reads the retrieved passages in its context window. The “augmentation” happens at inference, in the prompt, not through learned weights.
Any corpus, not just Wikipedia. The paper indexed Wikipedia. Modern systems index whatever documents you have: your own files, a product’s docs, a legal corpus.
A bigger pipeline. Chunking strategies, rerankers, query rewriting, and evaluation harnesses have grown up around the core idea, none of which the original paper needed because its retriever and generator were trained for the job.

So “RAG” today usually means the pattern the paper introduced, retrieve relevant text then generate an answer grounded in it, rather than the specific trained model it described. Knowing the difference is useful: when a paper or a product says “RAG”, it is worth asking whether they mean the training-free in-context pattern or something closer to the original end-to-end design.