Task-Aware Calibration: Provably Optimal Decoding in LLMs
For practitioners using LLMs for structured tasks, this provides a principled way to improve decoding reliability and performance.
LLM decoding is suboptimal when the model's predictive distribution is miscalibrated. The authors propose task calibration, which calibrates the distribution in a task-induced latent space, and show that Minimum Bayes Risk decoding on this calibrated distribution is optimal, consistently improving generation quality across tasks.
LLM decoding often relies on the model's predictive distribution to generate an output. Consequently, misalignment with respect to the true generating distribution leads to suboptimal decisions in practice. While a natural solution is to calibrate the model's output distribution, for LLMs, this is ill-posed at the combinatorially vast level of free-form language. We address this by building on the insight that in many tasks, these free-form outputs can be interpreted in a semantically meaningful latent structure, for example, discrete class labels, integers, or sets. We introduce task calibration as a paradigm to calibrate the model's predictive distribution in the task-induced latent space. We apply a decision-theoretic result to show that Minimum Bayes Risk (MBR) decoding on the task-calibrated latent distribution is the optimal decoding strategy on latent model beliefs. Empirically, it consistently improves generation quality across different tasks and baselines. We also introduce Task Calibration Error (TCE), an application-aware calibration metric that quantifies the excess loss due to miscalibration. Our work demonstrates that task calibration enables more reliable model decisions across various tasks and applications.