Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration
This addresses uncertainty calibration for deep learning models, offering a more expressive and accurate solution, though it is incremental as it builds on temperature scaling.
The paper tackles the problem of uncalibrated predictions in deep neural networks by introducing Parameterized Temperature Scaling (PTS), a post-hoc calibration method that computes prediction-specific temperatures via a neural network, and shows it consistently outperforms existing methods across various architectures, datasets, and metrics.
We address the problem of uncertainty calibration and introduce a novel calibration method, Parametrized Temperature Scaling (PTS). Standard deep neural networks typically yield uncalibrated predictions, which can be transformed into calibrated confidence scores using post-hoc calibration methods. In this contribution, we demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We generalize temperature scaling by computing prediction-specific temperatures, parameterized by a neural network. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics.