Confidence Calibration for Convolutional Neural Networks Using Structured Dropout
This addresses confidence calibration for users of convolutional neural networks in classification tasks, but it is incremental as it builds on existing dropout techniques.
The paper tackled the problem of poorly calibrated confidence estimates in convolutional neural networks using dropout, and found that structured dropout improves calibration by promoting model diversity, with empirical comparisons on SVHN, CIFAR-10, and CIFAR-100 datasets.
In classification applications, we often want probabilistic predictions to reflect confidence or uncertainty. Dropout, a commonly used training technique, has recently been linked to Bayesian inference, yielding an efficient way to quantify uncertainty in neural network models. However, as previously demonstrated, confidence estimates computed with a naive implementation of dropout can be poorly calibrated, particularly when using convolutional networks. In this paper, through the lens of ensemble learning, we associate calibration error with the correlation between the models sampled with dropout. Motivated by this, we explore the use of structured dropout to promote model diversity and improve confidence calibration. We use the SVHN, CIFAR-10 and CIFAR-100 datasets to empirically compare model diversity and confidence errors obtained using various dropout techniques. We also show the merit of structured dropout in a Bayesian active learning application.