Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results
This work addresses a critical issue for researchers and practitioners in AI safety and reliability by exposing spurious interactions in UQ evaluation, though it is incremental as it builds on existing UQ methods and metrics.
The paper tackled the problem of evaluating uncertainty quantification (UQ) in language models by showing that mutual biases, such as length biases in both UQ methods and correctness functions, systematically distort AUROC rankings, compromising benchmark integrity; empirical tests across 7 correctness functions, 4 datasets, 4 models, and 8 UQ methods confirmed this, with LM-as-a-judge methods identified as the least length-biased.
Uncertainty Quantification (UQ) in Language Models (LMs) is key to improving their safety and reliability. Evaluations often use metrics like AUROC to assess how well UQ methods (e.g., negative sequence probabilities) correlate with task correctness functions (e.g., ROUGE-L). We show that mutual biases--when both UQ methods and correctness functions are biased by the same factors--systematically distort evaluation. First, we formally prove that any mutual bias non-randomly skews AUROC rankings, compromising benchmark integrity. Second, we confirm this happens empirically by testing 7 widely used correctness functions, from lexical-based and embedding-based metrics to LM-as-a-judge approaches, across 4 datasets x 4 models x 8 UQ methods. Our analysis shows that length biases in correctness functions distort UQ assessments by interacting with length biases in UQ methods. We identify LM-as-a-judge methods as the least length-biased, offering a promising path for a fairer UQ evaluation.