When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
This reveals a critical vulnerability in current hallucination detection for LLMs, with broad implications for AI safety and reliability.
The paper tackles the problem of hallucinations in large language models caused by spurious correlations in training data, showing that existing detection methods like confidence-based filtering fail completely against these hallucinations, which persist even with model scaling and fine-tuning.
Despite substantial advances, large language models (LLMs) continue to exhibit hallucinations, generating plausible yet incorrect responses. In this paper, we highlight a critical yet previously underexplored class of hallucinations driven by spurious correlations -- superficial but statistically prominent associations between features (e.g., surnames) and attributes (e.g., nationality) present in the training data. We demonstrate that these spurious correlations induce hallucinations that are confidently generated, immune to model scaling, evade current detection methods, and persist even after refusal fine-tuning. Through systematically controlled synthetic experiments and empirical evaluations on state-of-the-art open-source and proprietary LLMs (including GPT-5), we show that existing hallucination detection methods, such as confidence-based filtering and inner-state probing, fundamentally fail in the presence of spurious correlations. Our theoretical analysis further elucidates why these statistical biases intrinsically undermine confidence-based detection techniques. Our findings thus emphasize the urgent need for new approaches explicitly designed to address hallucinations caused by spurious correlations.