Retrieval-Augmented Generation
Before answering, look up relevant passages in a document collection and paste them into the prompt, so the model answers from real sources instead of memory.
Also RAG · retrieval augmentation · vector search
Lewis et al., 2020
An open-book exam. Instead of answering from memory and maybe making things up, the model first flips to the right pages, then answers using what is written there.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
- AI Agents and Tool Use
- BERT
- Direct Preference Optimization
- Embedding
- Fine-Tuning
- Google Translate
- GPT
- Hallucination
- In-Context Learning
- Keyboard Autocomplete
- Large Language Model
- LLM Evaluation and Benchmarks
- Perplexity
- Pretraining
- Prompting
- RLHF
- Sampling and Temperature
- Scaling Laws
- Tokenization
- Vector Database
- word2vec
Uses / built on
Applies to
Techniques that apply here
How it works
A large language model knows only what was in its training data, and it fills gaps with plausible-sounding guesses. RAG gives it a library card.
Preparation (once):
- Split your documents into chunks of a few paragraphs.
- Compute an embedding for each chunk and store them in a vector index.
At question time:
- Embed the user’s question.
- Retrieve the chunks whose embeddings are closest (a nearest-neighbor search, see kNN), often combined with keyword search.
- Augment the prompt: “Answer using these passages: …”
- Generate the answer, ideally with citations to the passages used.
Update the documents and the answers update too, with no retraining. The weak spot is retrieval: if the right chunk is never found, the model is back to guessing. Much practical RAG work is about better chunking, hybrid search and re-ranking.
Everyday use cases
- Case 01
Company help desk
A chatbot answers employee questions from the internal handbook and policy documents, with links to the source.
- Case 02
Customer support
Answers grounded in the current product manuals, updated without retraining the model.
- Case 03
Research assistants
Search a collection of papers and summarize what they say about a question.
Strengths, weaknesses and when to use it
Strengths
- Reduces hallucination and allows citations.
- Knowledge updates by re-indexing documents, no retraining.
- Keeps private data out of model weights.
Weaknesses
- Only as good as retrieval. If the right passage is not found, the answer suffers.
- Chunking, indexing and ranking need tuning.
When to use itWhen answers must come from a specific, changing or private set of documents.
Check yourself
- Q01
What problem does RAG mainly address?
Show answer
B: Answers based on outdated or missing knowledge, and hallucination
- Q02
How are relevant passages typically found?
Show answer
A: By comparing embeddings of the question and of document chunks