CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment
This addresses the need for reliable uncertainty estimation in deploying neural networks in real-world applications, representing a novel method for a known bottleneck.
The paper tackles the problem of unreliable uncertainty estimation in neural networks by introducing CLUE, a method that aligns predicted uncertainty with observed error during training, achieving superior calibration quality and competitive predictive performance across various tasks without significant computational overhead.
Reliable uncertainty estimation is critical for deploying neural networks (NNs) in real-world applications. While existing calibration techniques often rely on post-hoc adjustments or coarse-grained binning methods, they remain limited in scalability, differentiability, and generalization across domains. In this work, we introduce CLUE (Calibration via Learning Uncertainty-Error Alignment), a novel approach that explicitly aligns predicted uncertainty with observed error during training, grounded in the principle that well-calibrated models should produce uncertainty estimates that match their empirical loss. CLUE adopts a novel loss function that jointly optimizes predictive performance and calibration, using summary statistics of uncertainty and loss as proxies. The proposed method is fully differentiable, domain-agnostic, and compatible with standard training pipelines. Through extensive experiments on vision, regression, and language modeling tasks, including out-of-distribution and domain-shift scenarios, we demonstrate that CLUE achieves superior calibration quality and competitive predictive performance with respect to state-of-the-art approaches without imposing significant computational overhead.