Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques
This work addresses the need for reliable confidence estimates in safety-critical applications like cancer prediction and autonomous driving, though it is incremental as it builds on existing temperature scaling methods.
The paper tackled the problem of poor confidence calibration in deep neural networks for image classification by introducing bin-wise temperature scaling and validation sample augmentation, achieving consistent improvements in calibration performance across various datasets and models.
The prediction reliability of neural networks is important in many applications. Specifically, in safety-critical domains, such as cancer prediction or autonomous driving, a reliable confidence of model's prediction is critical for the interpretation of the results. Modern deep neural networks have achieved a significant improvement in performance for many different image classification tasks. However, these networks tend to be poorly calibrated in terms of output confidence. Temperature scaling is an efficient post-processing-based calibration scheme and obtains well calibrated results. In this study, we leverage the concept of temperature scaling to build a sophisticated bin-wise scaling. Furthermore, we adopt augmentation of validation samples for elaborated scaling. The proposed methods consistently improve calibration performance with various datasets and deep convolutional neural network models.