Confidence Estimation Using Unlabeled Data
This addresses the issue of overconfidence for deploying neural networks in real-world applications, offering a novel approach for semi-supervised scenarios.
The paper tackles the problem of overconfidence in deep neural networks by proposing the first confidence estimation method for semi-supervised settings where most training labels are unavailable, achieving state-of-the-art performance on image classification and segmentation tasks.
Overconfidence is a common issue for deep neural networks, limiting their deployment in real-world applications. To better estimate confidence, existing methods mostly focus on fully-supervised scenarios and rely on training labels. In this paper, we propose the first confidence estimation method for a semi-supervised setting, when most training labels are unavailable. We stipulate that even with limited training labels, we can still reasonably approximate the confidence of model on unlabeled samples by inspecting the prediction consistency through the training process. We use training consistency as a surrogate function and propose a consistency ranking loss for confidence estimation. On both image classification and segmentation tasks, our method achieves state-of-the-art performances in confidence estimation. Furthermore, we show the benefit of the proposed method through a downstream active learning task. The code is available at https://github.com/TopoXLab/consistency-ranking-loss