HalluShift: Measuring Distribution Shifts towards Hallucination Detection in LLMs
This addresses the issue of LLMs generating incorrect but coherent information, which is a critical problem for users relying on accurate AI-generated content, though it appears incremental as it builds on existing detection methods.
The paper tackles the problem of hallucinations in Large Language Models by hypothesizing they stem from internal dynamics and introduces HalluShift to analyze distribution shifts in internal states and token probabilities, achieving superior performance on benchmark datasets.
Large Language Models (LLMs) have recently garnered widespread attention due to their adeptness at generating innovative responses to the given prompts across a multitude of domains. However, LLMs often suffer from the inherent limitation of hallucinations and generate incorrect information while maintaining well-structured and coherent responses. In this work, we hypothesize that hallucinations stem from the internal dynamics of LLMs. Our observations indicate that, during passage generation, LLMs tend to deviate from factual accuracy in subtle parts of responses, eventually shifting toward misinformation. This phenomenon bears a resemblance to human cognition, where individuals may hallucinate while maintaining logical coherence, embedding uncertainty within minor segments of their speech. To investigate this further, we introduce an innovative approach, HalluShift, designed to analyze the distribution shifts in the internal state space and token probabilities of the LLM-generated responses. Our method attains superior performance compared to existing baselines across various benchmark datasets. Our codebase is available at https://github.com/sharanya-dasgupta001/hallushift.