Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models
This addresses the challenge of trustworthy LLM deployment by providing a lightweight, broadly applicable hallucination detector, though it is incremental as it builds on existing uncertainty and complexity-based methods.
The paper tackles the problem of detecting hallucinations in large language models (LLMs) by proposing a training-free detector based on the complexity of the conditional distribution defined by the LLM and prompt, using optimal-transport distances between token embeddings to derive signals like AvgWD and EigenWD, with experiments showing competitive performance with uncertainty baselines across models and datasets.
Hallucinations in large language models (LLMs) remain a central obstacle to trustworthy deployment, motivating detectors that are accurate, lightweight, and broadly applicable. Since an LLM with a prompt defines a conditional distribution, we argue that the complexity of the distribution is an indicator of hallucination. However, the density of the distribution is unknown and the samples (i.e., responses generated for the prompt) are discrete distributions, which leads to a significant challenge in quantifying the complexity of the distribution. We propose to compute the optimal-transport distances between the sets of token embeddings of pairwise samples, which yields a Wasserstein distance matrix measuring the costs of transforming between the samples. This Wasserstein distance matrix provides a means to quantify the complexity of the distribution defined by the LLM with the prompt. Based on the Wasserstein distance matrix, we derive two complementary signals: AvgWD, measuring the average cost, and EigenWD, measuring the cost complexity. This leads to a training-free detector for hallucinations in LLMs. We further extend the framework to black-box LLMs via teacher forcing with an accessible teacher model. Experiments show that AvgWD and EigenWD are competitive with strong uncertainty baselines and provide complementary behavior across models and datasets, highlighting distribution complexity as an effective signal for LLM truthfulness.