Definition
Hallucination occurs because LLMs are probabilistic text generators that predict plausible next tokens based on patterns learned during training. They do not have a truth-verification mechanism—they generate text that sounds statistically appropriate for the context, whether or not it corresponds to reality. Hallucinations range from subtle factual errors (incorrect dates, slightly wrong statistics) to wholesale fabrications (fake citations, nonexistent regulations, invented case law). The danger is compounded by the model's confident tone, which makes hallucinations indistinguishable from accurate statements without independent verification.
Engineering Context
Hallucination is the primary reliability risk in production AI agents. Mitigation strategies: (1) RAG grounding—always retrieve source documents and instruct the model to cite them, (2) confidence scoring—require the model to rate its confidence and route low-confidence outputs to human review, (3) output validation—cross-check factual claims against source documents programmatically, (4) temperature 0 for factual tasks. Design agents to explicitly refuse rather than guess: instruct the model to say "I don't have information about this" rather than fabricate. For high-stakes domains (legal, medical, financial), implement mandatory citation requirements and automated citation verification as post-processing steps.
Related Terms
Building production AI agents?
We design and implement deterministic AI agent systems for enterprise teams.
Start Assessment