Skip to main content

Hallucination in Large Language Models

LLMs generate fluent, confident, wrong text. The taxonomy of hallucination types, the mechanistic causes, detection strategies, and mitigation techniques — and why this problem may be inherent to the architecture.

finished certainty: likely importance: 8

Hallucination in Large Language Models

If you use large language models for anything that matters, you’ve encountered hallucination: the model produces text that is fluent, confident, internally consistent, and wrong. Not wrong in the way a student is wrong—confused, tentative, visibly struggling—but wrong in the way a confident expert is wrong, with the kind of smooth authority that makes the error harder to catch. The model doesn’t hedge. It doesn’t say “I’m not sure.” It fabricates citations, invents facts, and confabulates entire histories with the same syntactic polish it applies to true statements.1

This is the reliability gap that separates “impressive demo” from “deployable system.” A model that hallucinates 5% of the time is usable for brainstorming and unusable for medical advice. A model that hallucinates 1% of the time is usable for first drafts and unusable for legal filings. The tolerance for hallucination varies by domain, but in high-stakes applications—healthcare, law, finance—even very low rates are unacceptable.

Taxonomy of Hallucination

Hallucinations come in distinct flavors, and understanding the taxonomy matters because different types have different causes and different mitigations.

Intrinsic hallucination occurs when the generated text contradicts the source material or the model’s own context. If you give the model a document saying “the contract expires in 2025” and the model summarizes it as “the contract expires in 2027,” that’s intrinsic hallucination. The model had the right information and produced the wrong output.

Extrinsic hallucination occurs when the generated text contains claims that cannot be verified from the source material—they may be true, false, or unfalsifiable. A model summarizing a research paper that adds a claim not present in the paper is producing extrinsic hallucination, regardless of whether the claim happens to be correct.2

Factual hallucination is the most commonly discussed type: the model asserts something about the world that is simply false. “Paris is the capital of Germany.” “GPT-4 was released in 2021.” “The study by Smith et al. (2019) showed that…” where no such study exists. This is what most people mean when they say a model “hallucinated.”

Faithfulness hallucination occurs in grounded tasks: the model fails to faithfully represent information it was given. This is the summarization problem—the model is supposed to compress, not invent—and it turns out to be surprisingly difficult to prevent.3

Why Do LLMs Hallucinate?

The causes of hallucination are multiple and interact:

The training objective doesn’t reward truth. LLMs are trained to predict the next token. A model that outputs “Paris is the capital of France” and a model that outputs “Paris is the capital of Germany” are both minimizing the same objective—they’re both producing fluent text that is likely given the preceding context. The training signal does not distinguish between true and false completions; it distinguishes between likely and unlikely ones. Truth and likelihood correlate—most of the training data is roughly accurate—but the correlation is imperfect, and the gaps are exactly where hallucinations live.4

Knowledge gaps and frequency effects. Models know more about frequent topics than rare ones. Hallucination rates are higher for obscure facts, recent events (not in the training data), and domains poorly represented in the pretraining corpus. Mallen et al. (2023) showed that LLM accuracy on factual questions is strongly correlated with the popularity of the entity in question—models are more accurate about Barack Obama than about a small-town mayor.

Compression and interference. A model with hundreds of billions of parameters stores trillions of facts, and those facts are not stored in discrete memory locations but in distributed, overlapping representations. Retrieval from these representations is approximate, and sometimes features from different facts interfere, producing chimeric outputs. A model might merge facts about two different people, two different events, or two different studies.

Decoding randomness. Standard generation uses temperature-based sampling, which introduces deliberate randomness. At temperature 0 (greedy decoding), models hallucinate less but produce repetitive, degenerate text. At higher temperatures, they’re more creative but more prone to hallucination. There is no temperature setting that simultaneously optimizes for both diversity and factuality.5

RLHF side effects. Ironically, the alignment training meant to make models more helpful may increase certain types of hallucination. RLHF rewards models for providing helpful, complete-seeming answers. A model that says “I don’t know” scores lower than a model that provides a plausible-sounding (but fabricated) answer. This creates a systematic pressure toward confident confabulation—exactly the kind of hallucination that’s hardest to detect.6

Detection Methods

Detecting hallucination is, in many ways, harder than generating it. The output is grammatically correct and topically relevant—it just happens to be false. Several approaches have been developed:

Self-consistency checking. Sample multiple responses to the same query and check for agreement. If the model gives different answers each time, at least some of them are hallucinations. Wang et al. (2023) formalized this as “self-consistency” prompting and showed it improves accuracy on reasoning tasks. The intuition is that facts are stable across samples but hallucinations vary.

Retrieval-based verification. Compare the model’s claims against a trusted knowledge base or retrieved documents. Retrieval-augmented generation (RAG) can both reduce hallucination (by grounding generation in retrieved evidence) and detect it (by checking outputs against sources). Systems like RARR (Retrofit Attribution using Research and Revision) retroactively attribute model claims to evidence and flag unsupported statements.

Uncertainty estimation. Models that can express calibrated uncertainty—“I’m 90% confident that…” and being right about 90% of the time—would naturally flag potential hallucinations. Current LLMs are poorly calibrated, especially after RLHF training, but techniques like conformal prediction and logit-based uncertainty estimation are improving.7

Natural language inference (NLI). Train a separate model to check whether the generated text is entailed by the source. This works reasonably well for summarization faithfulness but scales poorly to open-ended generation where there’s no single source document.

Mitigation Strategies

No single technique eliminates hallucination, but several reduce it substantially:

Retrieval-augmented generation (RAG). Ground the model’s generation in retrieved documents. Instead of asking “What are the side effects of drug X?” and trusting the model’s parametric knowledge, retrieve the drug’s documentation and ask the model to answer based on the retrieved text. This dramatically reduces hallucination for factual questions but doesn’t eliminate it—models can still misinterpret or ignore retrieved evidence.8

Chain-of-thought reasoning. Models that show their reasoning step-by-step hallucinate less on tasks that benefit from decomposition, like math and logic. The mechanism is plausible: intermediate steps create internal checkpoints that constrain the final answer. But chain-of-thought doesn’t prevent hallucination in the steps themselves—a model can reason fluently from a false premise.

Fine-tuning on verified data. Training on datasets where every claim has been verified reduces hallucination on similar domains. This is expensive and doesn’t generalize well—a model fine-tuned on verified medical data will still hallucinate about history.

Constitutional training. Constitutional AI includes honesty and accuracy among its constitutional principles, explicitly training the model to avoid hallucination. This helps but is bounded by the model’s ability to distinguish its own confident errors from its confident correct statements.

Post-generation verification. Run the model’s output through a verification pipeline—fact-checking, citation verification, consistency checks—before presenting it to the user. This is the “defense in depth” approach: accept that the model will hallucinate and catch the hallucinations after the fact. Tools like FActScore break generated text into atomic claims and verify each one against a knowledge source.

Is Hallucination Inherent?

There is a view—not yet proven but increasingly plausible—that some level of hallucination is inherent to the autoregressive language modeling paradigm. The argument goes: a model trained to predict the next token will always produce outputs that are plausible rather than true, because plausibility and truth are different things, and the training objective optimizes for the former.9

If this is correct, then hallucination is not a bug to be fixed but a fundamental limitation to be managed—through retrieval, verification, uncertainty quantification, and careful system design. The path forward is not “make models that don’t hallucinate” but “build systems around models that handle hallucination gracefully.” This is less satisfying but probably more realistic.

The practical implication is that any serious deployment of LLMs must include hallucination mitigation as a first-class concern, not an afterthought. The model is the engine, not the car. And an engine without brakes, steering, and seatbelts is not something you want on the road.

Further Reading