The Energy of Falsehood: Detecting Hallucinations via Diffusion Model Likelihoods
This addresses the critical issue of LLM hallucinations for AI safety and reliability, though it builds incrementally on existing diffusion and verification methods.
The paper tackles the problem of detecting hallucinations in Large Language Models by proposing DiffuTruth, an unsupervised framework that uses diffusion model likelihoods to measure factual stability. The method achieves state-of-the-art unsupervised AUROC of 0.725 on FEVER and outperforms baselines by over 4% on HOVER in zero-shot generalization.
Large Language Models (LLMs) frequently hallucinate plausible but incorrect assertions, a vulnerability often missed by uncertainty metrics when models are confidently wrong. We propose DiffuTruth, an unsupervised framework that reconceptualizes fact verification via non equilibrium thermodynamics, positing that factual truths act as stable attractors on a generative manifold while hallucinations are unstable. We introduce the Generative Stress Test, claims are corrupted with noise and reconstructed using a discrete text diffusion model. We define Semantic Energy, a metric measuring the semantic divergence between the original claim and its reconstruction using an NLI critic. Unlike vector space errors, Semantic Energy isolates deep factual contradictions. We further propose a Hybrid Calibration fusing this stability signal with discriminative confidence. Extensive experiments on FEVER demonstrate DiffuTruth achieves a state of the art unsupervised AUROC of 0.725, outperforming baselines by 1.5 percent through the correction of overconfident predictions. Furthermore, we show superior zero shot generalization on the multi hop HOVER dataset, outperforming baselines by over 4 percent, confirming the robustness of thermodynamic truth properties to distribution shifts.