Credal Transformer: A Principled Approach for Quantifying and Mitigating Hallucinations in Large Language Models
This addresses the issue of unreliable AI outputs for users by providing a principled approach to mitigate hallucinations, though it is incremental as it builds on existing Transformer architectures.
The paper tackles the problem of hallucinations in Large Language Models by introducing the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism to quantify uncertainty and reduce confident errors, achieving significant reductions in errors on unanswerable questions through abstention.
Large Language Models (LLMs) hallucinate, generating factually incorrect yet confident assertions. We argue this stems from the Transformer's Softmax function, which creates "Artificial Certainty" by collapsing ambiguous attention scores into a single probability distribution, discarding uncertainty information at each layer. To fix this, we introduce the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism (CAM) based on evidential theory. CAM produces a "credal set" (a set of distributions) instead of a single attention vector, with the set's size directly measuring model uncertainty. We implement this by re-conceptualizing attention scores as evidence masses for a Dirichlet distribution: sufficient evidence recovers standard attention, while insufficient evidence yields a diffuse distribution, representing ambiguity. Empirically, the Credal Transformer identifies out-of-distribution inputs, quantifies ambiguity, and significantly reduces confident errors on unanswerable questions by abstaining. Our contribution is a new architecture to mitigate hallucinations and a design paradigm that integrates uncertainty quantification directly into the model, providing a foundation for more reliable AI.