Retrieval-augmented generation (RAG) is a technique that fetches relevant passages from a body of text first, then feeds them to a language model so its answer is grounded in real source material rather than the model's memory.
Why it matters
A language model on its own answers from patterns baked into its training data. Ask it about a specific document and it has no way to actually see that document - so it guesses, and guessing is where hallucination comes from. RAG closes that gap. By putting the relevant text in front of the model at answer time, it turns "what does the model vaguely remember?" into "what does this source actually say?"
That shift does two things at once. It cuts hallucination, because the model is working from text it can read rather than statistics it half-recalls. And it makes citation possible, because every claim can be traced back to the exact passage that was retrieved to support it.
How it works
A RAG pipeline has two stages:
- Retrieve. The system finds the passages most relevant to your question, usually with semantic search over the document - matching on meaning, not just keywords.
- Generate. Those passages are passed to the model as context, and it composes an answer constrained to what they say. This is the source-grounding step.
The quality of the answer depends heavily on the first stage: retrieve the wrong passages and even a perfect model produces a confident, wrong response.
Plain LLM vs. RAG
The clearest way to see what RAG adds is to compare a model answering with nothing but its training data against the same model answering with retrieved passages in front of it.
| Plain LLM | RAG | |
|---|---|---|
| Where the facts come from | The model's training data - a compressed memory of text it once read | Passages retrieved from your actual document at answer time |
| Can it see your specific document? | No - unless the whole thing fits in the prompt already | Yes - retrieval finds and injects the relevant parts |
| Citable to an exact source | No - there's nothing to point back to | Yes - every claim can be traced to the retrieved passage |
| Risk of hallucination | Higher - the model fills gaps with plausible-sounding guesses | Lower - the model is constrained to what it was actually shown |
| Stays current with new information | No - frozen at training time | Yes - retrieval pulls from whatever is indexed right now |
That's the entire value proposition of RAG in one row: it turns "what does the model recall?" into "what does this document actually say?" - which is also what makes an answer worth trusting instead of just plausible.
How Sidenote uses it
Sidenote runs retrieval over the document you're already reading - a PDF, a Confluence page, a research paper - rather than a separate upload. It pulls the passages that match your question, grounds the answer in them, and attaches a citation that scrolls to the source. Any claim it can't tie back to a retrieved passage is dropped before you see it, so every answer points at the sentence it came from.