Teaching Language Models to Faithfully Express their Uncertainty
This addresses the faithfulness gap in LLMs for users relying on accurate uncertainty communication, though it is an incremental improvement over existing fine-tuning methods.
The paper tackles the problem of large language models (LLMs) miscommunicating their uncertainty by introducing Faithful Uncertainty Tuning (FUT), a fine-tuning approach that teaches LLMs to express uncertainty faithfully without altering answer distributions, resulting in a substantial reduction in the faithfulness gap while preserving QA accuracy.
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in ways that do not reflect this variability. This conveys unfaithful information about the uncertain state of the LLMs' knowledge, creating a faithfulness gap that affects even strong LLMs. We introduce Faithful Uncertainty Tuning (FUT): a fine-tuning approach that teaches instruction-tuned LLMs to express uncertainty faithfully without altering their underlying answer distribution. We construct training data by augmenting model samples with uncertainty hedges (i.e. verbal cues such as 'possibly' or 'likely') aligned with sample consistency, requiring no supervision beyond the model and a set of prompts. We evaluate FUT on open-domain question answering (QA) across multiple models and datasets. Our results show that FUT substantially reduces the faithfulness gap, while preserving QA accuracy and introducing minimal semantic distribution shift. Further analyses demonstrate robustness across decoding strategies, choice of hedgers, and other forms of uncertainty expression (i.e. numerical). These findings establish FUT as a simple and effective way to teach LLMs to communicate uncertainty faithfully.