LLM Hallucination: Causes and Mitigations
Reports define LLM hallucinations, analyze causes from pretraining and fine-tuning, and survey detection and mitigation techniques. - Hallucination defined as fabricated content - Two types: context hallucination and external hallucination - Pretraining data contains errors and outdated facts - Fine-tuning can increase hallucination when forcing new knowledge - Gekhman et al. 2024: models learn known facts faster than new facts - Models that overfit limited new facts hallucinate more - Detection: retrieval-backed verification and FactScore atomic checking - Safe methods: SAFE multi-step search and SelfCheckGPT sampling consistency - Mitigation: RAG, RARR two-step retrieval+revise, Self-RAG decisions during generation - Verification: CoVe chain-of-verification validates facts separately - Sampling: factual-nucleus sampling reduces late-answer hallucinations - Fine-tuning: FLAME uses higher factual-quality data and FactScore rewards - PM interview guidance emphasizes understanding LLM limits and hallucination risk
Key insights
Hallucination defined and categorized
The posts define hallucination as model-generated content that does not align with context or external facts, categorizing it into context hallucination (mismatch with provided materials) and external hallucination (contradicting real-world knowledge or failing to admit uncertainty). The definition emphasizes that 'hallucination' has been overused as a general term for model error.
Training data and fine-tuning dynamics cause hallucinations
Pretraining on large web corpora exposes models to outdated and erroneous content; post-pretraining fine-tuning can worsen hallucination when models are forced to learn new, sparsely represented facts. Gekhman et al. 2024 found models learn 'known' knowledge faster than new facts, and forcing new knowledge increases hallucination rates.
Detection techniques: retrieval, atomic checks, and consistency
Detection approaches include retrieval-augmented verification (checking claims against knowledge bases), FactScore atomic-fact checking, SAFE multi-step evidence searches, and SelfCheckGPT sampling-consistency checks that compare multiple model outputs to detect unstable facts.
Mitigations: RAG, verification chains, sampling, and fine-tuning quality
Reported mitigations include RAG and RARR (research then revise), Self-RAG for model-driven retrieval decisions, CoVe chain-of-verification to test claims via separate validation questions, factual-nucleus sampling to reduce late-answer errors, and FLAME-style fine-tuning that uses higher factual-quality data and FactScore as reward to reduce hallucination.
Product management must account for LLM limits and hallucination risk
Interview-style guidance for AI product managers emphasizes assessing LLM capability boundaries, choosing high-value AI features, defining AI limits, designing fallbacks for hallucinations (e.g., provenance, human-in-the-loop), and using metrics and monitoring to diagnose and mitigate hallucination-related user complaints.