HAVE: Head-Adaptive Gating and ValuE Calibration for Hallucination Mitigation in Large Language Models
This addresses hallucinations in LLMs for real-world trustworthy generation, but it is incremental as it builds on existing attention mechanisms.
The paper tackled the problem of hallucinations in large language models during retrieval-augmented or long-context generation by introducing HAVE, a parameter-free decoding framework that reduces hallucinations and outperforms baselines like DAGCD with modest overhead.
Large Language Models (LLMs) often produce hallucinations in retrieval-augmented or long-context generation, even when relevant evidence is present. This stems from two issues: head importance is treated as input-agnostic, and raw attention weights poorly reflect each token's true contribution. We present HAVE (Head-Adaptive Gating and ValuE Calibration), a parameter-free decoding framework that directly addresses both challenges. HAVE introduces head-adaptive gating, which performs instance-level soft reweighing of attention heads, and value calibration, which augments attention with the magnitude of value vectors to approximate write-back contribution. Together, these modules construct token-level evidence aligned with model updates and fuse it with the LM distribution through a lightweight uncertainty-scaled policy. HAVE requires no finetuning and operates in a single forward pass, making it efficient and broadly applicable. Experiments across multiple QA benchmarks and LLM families demonstrate that HAVE consistently reduces hallucinations and outperforms strong baselines, including DAGCD, with modest overhead. The framework is transparent, reproducible, and readily integrates with off-the-shelf LLMs, advancing trustworthy generation in real-world settings.