The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
For practitioners deploying LLMs, this work provides a practical method to obtain better-calibrated and more informative confidence estimates, enhancing trust in model outputs.
The paper addresses poor calibration in LLMs, which often overconfidently mispredict. They propose ACUTE, an activation-based confidence estimation protocol, and a new metric EURO that balances calibration and informativeness. ACUTE outperforms baselines on EURO across 3 tasks and 6 models, improving calibration and trustworthiness.
As language models improve and become increasingly deployed to solve a variety of tasks, trustworthiness becomes essential. Calibration is a good proxy for trust: well-calibrated confidence estimates help inform the risk versus reward tradeoff when trusting a specific model output. Unfortunately, even as models improve, they remain poorly calibrated, often biasing towards overconfidence. Additionally, calibration can be gamed: a policy that always predicts the base rate is perfectly calibrated, but completely uninformative. To resolve this, we develop a new metric, expected utility renormalized by the oracle (EURO), that balances calibration and informativeness. We also propose a general-purpose activation-based confidence, utility, and trust estimation protocol (ACUTE) to appropriately adjudicate uncertainty. The ACUTE protocol provides flexible, sample-efficient, and compute-efficient confidence estimators for 3 tasks including multiple choice question answering, tool-calling, and scientific document summarization across 6 models from 4 model families. ACUTE outperforms strong baselines on EURO, while maintaining low calibration error. Taken together, our work shows that equipping LLMs with the ACUTE protocol can improve calibration, utility, and trustworthiness in numerous settings.