Short answer
A hallucination is output that is fluent, plausible and false: an invented citation, a policy the company never wrote, a confidently wrong figure. It occurs because a language model generates the most likely continuation rather than checking a source, so a wrong answer is produced with exactly the same fluency as a right one.
Also called: AI hallucination, Confabulation
Hallucination is a property of how generation works, not a defect awaiting a patch. A model that always produces a likely-looking continuation has no internal test for truth, and the same mechanism that lets it write a sensible sentence about a topic it was never taught is the mechanism that lets it invent a case citation. Newer models do it less often and the rate can be pushed low — no current technique reduces it to zero, and a vendor claiming otherwise is describing a hope.
Mitigation is therefore a design problem. Ground answers in retrieved documents and show the source so a reader can check it. Constrain the task, because an assistant answering from six policy pages fails far less often than one asked about anything at all. Give the system an explicit way to decline. Then decide which outputs a person must approve before they reach a customer, which is a business judgement about risk rather than a technical setting: a drafted social caption and a quoted price do not need the same scrutiny.
Common questions
Can hallucinations be eliminated completely?
No. They can be made rare and, more usefully, made visible. Grounding answers in your own documents, showing the source passage, narrowing what the system is allowed to answer, and routing high-stakes output through a person are the levers that work. Designing on the assumption that the model is occasionally wrong is safer than shopping for one that never is.
Is it safe to let AI answer customer questions directly?
It depends on the cost of a wrong answer. Opening hours, service scope and delivery areas are low-risk and easy to ground in a document. Prices with conditions, medical or legal guidance, and anything that creates an obligation deserve a human check or a scripted response rather than generated text.
Related terms
Large Language Model (LLM)
A large language model is a neural network trained on large volumes of text to predict likely continuations, which lets it write, summarise, classify and translate without being programmed for each task.
Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation is a technique that searches a private collection of documents for passages relevant to a question, then passes those passages to a language model as context for its answer.
Guardrails (AI)
Guardrails are the controls that constrain what an AI system may say and do: topic limits, validation of output against an expected structure, blocked actions, spending caps, and filters on both the incoming request and the generated response.
Human-in-the-Loop
Human-in-the-loop is a design pattern where an AI system prepares work and a person approves, edits or rejects it before it takes effect: drafted replies queued for a one-click send, extracted invoice fields shown for confirmation.
Reading definitions because you are scoping a project? Skip ahead and just ask.