Prompt-Guided Internal States for Hallucination Detection of Large Language Models
This addresses the issue of LLMs generating factually incorrect responses, which is critical for improving reliability in applications like content generation and fact-checking, though it is incremental as it builds on existing detection methods.
The paper tackles the problem of hallucination detection in large language models (LLMs) by enhancing cross-domain generalization using only in-domain data, resulting in a framework called PRISM that significantly improves existing methods' performance across different domains.
Large Language Models (LLMs) have demonstrated remarkable capabilities across a variety of tasks in different domains. However, they sometimes generate responses that are logically coherent but factually incorrect or misleading, which is known as LLM hallucinations. Data-driven supervised methods train hallucination detectors by leveraging the internal states of LLMs, but detectors trained on specific domains often struggle to generalize well to other domains. In this paper, we aim to enhance the cross-domain performance of supervised detectors with only in-domain data. We propose a novel framework, prompt-guided internal states for hallucination detection of LLMs, namely PRISM. By utilizing appropriate prompts to guide changes to the structure related to text truthfulness in LLMs' internal states, we make this structure more salient and consistent across texts from different domains. We integrated our framework with existing hallucination detection methods and conducted experiments on datasets from different domains. The experimental results indicate that our framework significantly enhances the cross-domain generalization of existing hallucination detection methods.