Attention Head Embeddings with Trainable Deep Kernels for Hallucination Detection in LLMs
This addresses the problem of hallucination detection in LLMs for users relying on accurate outputs, offering a scalable solution, though it is incremental as it builds on distributional analysis methods.
The paper tackles detecting hallucinations in large language models by analyzing probabilistic divergence between prompt and response hidden-state distributions, finding that hallucinated responses show smaller deviations, and proposes a model-intrinsic method with deep learnable kernels that achieves state-of-the-art performance on benchmarks.
We present a novel approach for detecting hallucinations in large language models (LLMs) by analyzing the probabilistic divergence between prompt and response hidden-state distributions. Counterintuitively, we find that hallucinated responses exhibit smaller deviations from their prompts compared to grounded responses, suggesting that hallucinations often arise from superficial rephrasing rather than substantive reasoning. Leveraging this insight, we propose a model-intrinsic detection method that uses distributional distances as principled hallucination scores, eliminating the need for external knowledge or auxiliary models. To enhance sensitivity, we employ deep learnable kernels that automatically adapt to capture nuanced geometric differences between distributions. Our approach outperforms existing baselines, demonstrating state-of-the-art performance on several benchmarks. The method remains competitive even without kernel training, offering a robust, scalable solution for hallucination detection.