RAG
The most-built system in applied AI — and the one most often described too shallowly.
Prerequisites
- Embeddings and ANN Indexes — retrieval is this section's engine.
- Tokens & Context Windows — the context budget is what chunking is negotiating over.
- Hallucination — the problem RAG is answering.
Why this section exists
An LLM's weights froze at a training cutoff and never contained your private data. You need it to answer from documents it has never seen, and to show its work. You have two options: change the weights (slow, expensive, bad at facts, no per-user access control) or change the prompt. RAG is the second option, done seriously.
Everyone can say "retrieve documents, put them in the prompt." Everything that actually matters sits one level below that, on a single insight:
The sentence to have ready
Almost every RAG failure is a retrieval failure. If the right chunk never reaches the prompt, no model, no prompt wording, and no amount of temperature tuning can save the answer. So the engineering effort — and the evaluation effort — goes into retrieval quality, not prompt polish.
Everything in this section follows from taking that sentence seriously.
What's in here
| Page | The question it answers |
|---|---|
| The RAG Pipeline | The end-to-end picture — this is the "explain RAG" answer |
| Ingestion & Chunking | Parsing real documents, chunking strategies, the size tradeoff, metadata design |
| Retrieval Patterns | Query rewriting, HyDE, multi-query, parent-child, contextual retrieval |
| Advanced RAG | Agentic RAG and graph RAG — when the extra machinery pays for itself |
| RAG Evaluation | Faithfulness, context precision/recall, golden datasets, LLM-as-judge and its biases |
| Production RAG | Freshness, document lifecycle, per-user ACLs, cost, latency, graceful degradation |
Read in that order. Evaluation before production is deliberate: you cannot productionize what you cannot measure, and "we shipped it and it felt better" is not a result.
Where this connects
- Agents — hand retrieval to the model as a tool and RAG becomes agentic RAG; the loop replaces the fixed pipeline.
- Security — retrieved content is untrusted input, which makes RAG a prompt-injection surface and a data-leakage surface at once.
- Design: Enterprise RAG — all of this, assembled into one system with explicit numbers.