Diversity regularization in deep ensembles
This work addresses the need for reliable uncertainty estimates in deep learning models, which is crucial for applications requiring trustworthy predictions, but it appears incremental as it builds on existing ensemble methods.
The authors tackled the problem of poor calibration in deep neural networks by proposing a diversity regularization strategy for training deep ensembles, which improved calibration while maintaining similar prediction accuracy.
Calibrating the confidence of supervised learning models is important for a variety of contexts where the certainty over predictions should be reliable. However, it has been reported that deep neural network models are often too poorly calibrated for achieving complex tasks requiring reliable uncertainty estimates in their prediction. In this work, we are proposing a strategy for training deep ensembles with a diversity function regularization, which improves the calibration property while maintaining a similar prediction accuracy.