Short answer
Retrieval-augmented generation is a technique that searches a private collection of documents for passages relevant to a question, then passes those passages to a language model as context for its answer. A support assistant built this way quotes your current refund policy because the policy text was retrieved and supplied, not memorised.
Also called: RAG, Grounded generation
The pipeline has three stages. Documents are split into chunks and converted into numeric representations, called embeddings, that place similar meanings near one another; a question is converted the same way and used to find the nearest chunks; those chunks are placed in the prompt with an instruction to answer from them. Because the source text sits in the prompt, the system can point to the document an answer came from, which is what makes the output checkable.
Retrieval quality is where these systems succeed or fail, and the failure is quiet. If the search returns the wrong three paragraphs, the model answers confidently from the wrong three paragraphs. The work is therefore housekeeping: sensible chunk sizes, clean source documents, a way to retire superseded versions, and evaluation against a fixed list of real questions. Grounding reduces invention substantially. It does not remove it.
Common questions
Is RAG better than fine-tuning?
For factual company knowledge, usually yes. Retrieval handles information that changes, can be updated by editing a document, and lets an answer cite its source. Fine-tuning is the better fit for teaching a model a consistent format, tone or classification behaviour. The two are often combined rather than chosen between.
What documents should we start with?
The ones your team already answers questions from: price lists, service scope, refund and warranty terms, delivery coverage, and the standing replies sitting in your inbox. Start with a small, current set. A large archive holding three versions of the same policy produces worse answers than twenty accurate pages.
Where this comes up in our work
Related terms
Large Language Model (LLM)
A large language model is a neural network trained on large volumes of text to predict likely continuations, which lets it write, summarise, classify and translate without being programmed for each task.
Hallucination (AI)
A hallucination is output that is fluent, plausible and false: an invented citation, a policy the company never wrote, a confidently wrong figure.
Fine-Tuning
Fine-tuning is further training of an existing language model on a set of example inputs and outputs, so it adopts a specific style, format or classification behaviour by default.
AI Agent
An AI agent is a system that uses a language model to decide which actions to take, then takes them through connected tools: reading a database, sending an email, updating a booking record.
Reading definitions because you are scoping a project? Skip ahead and just ask.