Hallucination
A hallucination is an answer from a language model that sounds right and is not: an invented figure, a feature that does not exist, a plausible number nobody checked. It is not a bug, it is a direct consequence of how a model works — it produces the most probable continuation, not the verified one.
What is hallucination?
A hallucination is an answer from a language model that sounds right and is not: an invented figure, a feature that does not exist, a plausible number nobody checked. It is not a bug, it is a direct consequence of how a model works — it produces the most probable continuation, not the verified one.
Also: confabulation · made-up answer
Not a one-off bug waiting for a patch
Hallucinations get discussed as a defect the next version will fix. Producing plausible text and inventing are the same capability, so what you can do is not eliminate them but corner them: limit where the system gets its data and make it recognise when it does not have any.
Why it matters
What changes in a SaaS
In support a hallucination is not a curiosity: it is a customer acting on something false. If the system tells them their plan includes something it does not, or gives an invented refund date, the cost is not the ticket — it is the complaint afterwards and the trust that goes with it.
Hallucination in detail
How to reduce them
By forcing the system to answer only from passages retrieved from your documentation and data looked up in your systems, rather than from what the model seems to remember. That is the whole point of RAG.
The right answer when it does not know
"I do not have that, let me get someone from the team." A system that says this is more useful than one that is right 95% of the time and invents the other 5% with equal confidence, because the second one forces you to check everything.
Where they hurt most
In numbers and conditions: prices, deadlines, limits, coverage. Those are the answers customers act on, and the easiest to sound plausible while being false. Those should always come from a lookup, never from the model memory.
How Intake handles it
Questions about hallucination
Can hallucinations be eliminated?
Not entirely: they follow from how a language model works. You corner them by limiting where the data comes from and designing the system to recognise when it has none.
Why does a model invent things so confidently?
Because it produces the most plausible continuation, and a well-formed false sentence is as plausible as a true one. The model does not distinguish between them — that distinction has to come from outside.
Related terms
A term on its own is only half understood. These come up in the same conversation.
LLM — Language model
A language model is a system trained on enormous amounts of text that, given a fragment, predicts how it continues. Writing, summarising, translating and holding a conversation all come out of that one simple capability: they are all ways of continuing a text plausibly.
RAG — Retrieval-augmented generation
Retrieval-augmented generation, or RAG, is the technique of finding relevant information in your own sources and handing it to a language model so it writes the answer with that. The model does not memorise the content: it looks it up at answer time.
AI agent
An AI agent is a system that, beyond generating text, reads data and takes actions to finish a task. In support that means it checks the actual state of the customer account before answering, and when the answer involves doing something, it does it instead of explaining how.
Chatbot
A chatbot is a program that holds a written conversation with a user. In support, current ones index the company documentation and answer from it; earlier ones followed decision trees written by hand.
Embedding
An embedding is a text represented as a list of numbers that encodes its meaning. Two texts that say the same thing with different words produce similar lists, and that numeric closeness is what makes it possible to search by meaning rather than by exact match.
Tool calling
Tool calling is the mechanism by which a language model can run functions you define instead of only writing. You describe the available tools, the model decides which one to use and with what arguments, and it receives the result to fold into its answer.
