An Uncertainty-aware Loss Function for Training Neural Networks with Calibrated Predictions
This work addresses the need for calibrated uncertainty estimates in neural networks, which is crucial for enhancing trust in AI predictions, though it is incremental as it builds on existing Monte Carlo Dropout techniques.
The authors tackled the problem of uncertainty quantification in deep learning by proposing two new loss functions that combine cross entropy with Expected Calibration Error and Predictive Entropy, resulting in a calibrated Monte Carlo Dropout method that reduces overlap between uncertainty distributions for correct and incorrect predictions without sacrificing overall performance.
Uncertainty quantification of machine learning and deep learning methods plays an important role in enhancing trust to the obtained result. In recent years, a numerous number of uncertainty quantification methods have been introduced. Monte Carlo dropout (MC-Dropout) is one of the most well-known techniques to quantify uncertainty in deep learning methods. In this study, we propose two new loss functions by combining cross entropy with Expected Calibration Error (ECE) and Predictive Entropy (PE). The obtained results clearly show that the new proposed loss functions lead to having a calibrated MC-Dropout method. Our results confirmed the great impact of the new hybrid loss functions for minimising the overlap between the distributions of uncertainty estimates for correct and incorrect predictions without sacrificing the model's overall performance.