Skip to content

Hallucination

Adrià Castany 2 min

What is a hallucination?

A hallucination is an answer from a language model that sounds right and is not. An invented figure, a feature that does not exist, a plausible number nobody checked. It is a direct consequence of how a model works. It produces the most probable continuation, not the verified one.

Also called

confabulation, made-up answer

Topic
AI in support
On this page
  1. Not a one-off bug waiting for a patch
  2. Why it matters
  3. How to reduce them
  4. The right answer when it does not know
  5. Where they hurt most
  6. How Intake handles it
  7. Frequently asked questions

Not a one-off bug waiting for a patch

Hallucinations get discussed as a defect the next version will fix.

Producing plausible text and inventing are the same capability, so you cannot eliminate them, but you can corner them. Limit where the system gets its data and make it recognize when it does not have any.

Why it matters

In support, a hallucination is a customer acting on something false.

If the system tells them their plan includes something it does not, or gives an invented refund date, what it costs is the complaint afterwards and the trust that goes with it, far more than the ticket.

How to reduce them

They are not eliminated, they are cornered.

Four measures do most of the work.

  • Forcing the system to answer only from passages retrieved from the company’s own documentation.
  • Giving it a way out when it lacks the fact, which is saying so and handing the conversation to a person.
  • Taking figures, deadlines and conditions from a source, never from the model’s wording.
  • Reviewing real conversations and testing the hard cases before every change.

The right answer when it does not know

"I do not have that, let me get someone from the team."

A system that says this is more useful than one that is right 95% of the time and invents the other 5% with equal confidence, because the second one forces you to check everything.

Where they hurt most

In numbers and conditions.

Prices, deadlines, limits, coverage. Those are the answers customers act on, and the easiest to sound plausible while being false. Those should always come from a lookup, never from the model’s memory.

How Intake handles it

Frequently asked questions

Can hallucinations be eliminated?

Not entirely. They follow from how a language model works. You corner them by limiting where the data comes from and designing the system to recognize when it has none.

Why does a model invent things so confidently?

Because it produces the most plausible continuation, and a well-formed false sentence is as plausible as a true one. The model does not distinguish between them; that distinction has to come from outside.

What is the difference between a hallucination and an error?

An error tends to show, because the answer does not fit. A hallucination is well written, sounds confident and is only discovered by checking the fact. That is why it does more damage.

Glossary

Keep reading

Glossary

Related terms

See the full glossary

Add it without dropping what you use

15 minutes, no commitment