Retrieval-augmented generation means finding the documents relevant to a question and putting them in front of the model, so it answers from your content rather than from memory. It is the standard way to make a model reliable about facts it was never trained on.
Ahead of time: split your documents into chunks, embed each chunk, and store the vectors. At question time: embed the question, retrieve the closest chunks, paste them into the prompt with an instruction to answer from them and cite which one, and return the model's answer alongside its sources.
Everything hard about RAG lives in the retrieval half. Getting the model to use supplied text is easy; getting the right text supplied is the engineering.
Because facts change and weights do not. With retrieval, updating an answer means updating a document; with fine-tuning it means another training run. Retrieval also produces citations, so a reader can check the claim, and it can enforce permissions per user in a way baked-in knowledge simply cannot.
| Approach | What it changes | Updating it | Reach for it when |
|---|---|---|---|
| Prompting | What the model is asked, and nothing else | Instant — edit the text | The knowledge is already in the model and only needs shaping |
| RAG | What the model can see at answer time | Reindex the documents | Answers must be grounded in facts the model never saw, and those facts change |
| Fine-tuning | How the model behaves by default | Retrain the model | A format, tone or task pattern must hold without being restated every time |
Assuming that once documents are in the prompt the answer must be correct. If retrieval returns nothing relevant, most models will answer anyway from general knowledge — so instruct explicitly that "the documents do not cover this" is an acceptable answer, and check that it is ever actually given.
The other is treating chunking as a detail. Splitting mid-sentence, discarding the heading that gave a paragraph its meaning, or using chunks so large that the useful line is buried will cap quality no matter how good the model is. Measure retrieval on its own — did the right chunk come back — before blaming the generation step.
No application to be approved and no card to enter. Pick a name and a password, choose the rooms that fit, and say what you are building.
