Hallucination
When an AI model states false information — an invented policy, step, or citation — with the same fluent confidence it uses for correct answers.
Hallucination rate = 1 - (responses fully grounded in source / total responses generated)A hallucination is AI output that is fluent, confident, and wrong — text the model presents as fact without any grounding in a real source. In support this looks like a bot inventing a refund window, describing a settings menu that doesn't exist, or citing a help article nobody ever wrote. The tell is that a fabrication arrives in exactly the same authoritative tone as a correct answer, so the customer has no signal to distrust it.
The standard way to quantify it is to ground the model in a source document and count how often it asserts claims the source doesn't support. On that grounded-summarization test the best current models sit near 1-2% while the weakest run above 20%. Retrieval-augmented generation (RAG) — feeding the model your own knowledge base — sharply lowers the rate but never drives it to zero.
The liability is real. In Moffatt v. Air Canada, a small-claims tribunal held the airline responsible after its chatbot invented a bereavement-fare policy that contradicted the airline's actual rules, ordering it to pay the customer and flatly rejecting the argument that the bot was a separate entity from the company.
What it hides: a single headline hallucination rate hides that the number is almost entirely task-dependent — grounded summarization can stay under 2% while open-ended or citation-heavy answers reach up to 86% of generated facts. It also hides the distribution: what matters is not the average but the tail — the rare, confident lie on a high-stakes question (a refund, a security step, a dosage) that no customer thinks to double-check.