Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Rate
This provides a practical solution for enhancing trustworthiness in LLM responses, particularly in API-constrained deployments like QA and RAG systems, though it is incremental as it builds on existing uncertainty metrics.
The paper tackles the problem of hallucinations in black-box LLM outputs for QA tasks by introducing a one-shot detection method using token-level entropy production rate and supervised learning, achieving significant improvements over baseline metrics across diverse datasets and LLMs while requiring only limited log-probability data.
Hallucinations in Large Language Model (LLM) outputs for Question Answering (QA) tasks critically undermine their real-world reliability. This paper introduces an applied methodology for robust, one-shot hallucination detection, specifically designed for scenarios with limited data access, such as interacting with black-box LLM APIs that typically expose only a few top candidate log-probabilities per token. Our approach derives uncertainty indicators directly from these readily available log-probabilities generated during non-greedy decoding. We first derive an Entropy Production Rate (EPR) metric that offers baseline performance, later augmented with supervised learning. Our learned model uses features representing the entropic contributions of the accessible top-ranked tokens within a single generated sequence, requiring no multiple query re-runs. Evaluated across diverse QA datasets and multiple LLMs, this estimator significantly improves hallucination detection over using EPR alone. Crucially, high performance is demonstrated using only the typically small set of available log-probabilities (e.g., top <10 per token), confirming its practical efficiency and suitability for these API-constrained deployments. This work provides a readily deployable technique to enhance the trustworthiness of LLM responses from a single generation pass in QA and Retrieval-Augmented Generation (RAG) systems, with its utility further demonstrated in a finance framework analyzing responses to queries on annual reports from an industrial dataset.