Consistent Relative Confidence and Label-Free Model Selection for Convolutional Neural Networks
This addresses the challenge of model selection in scenarios where labeling is costly or unavailable, offering a practical solution for image classification tasks.
The paper tackles the problem of selecting the best convolutional neural network model for image classification without labeled data, proposing a method based on consistent relative confidence that shows effectiveness and efficiency on benchmark datasets.
In this paper, we are concerned with image classification with deep convolutional neural networks (CNNs). We focus on the following question: given a set of candidate CNN models, how to select the right one with the best generalization property for the current task? Current model selection methods all require access to a batch of labeled data for computing a pre-specified performance metric, such as the cross-entropy loss, the classification error rate and the negative log-likelihood. In many practical cases, labels are not available in time as labeling itself is a time-consuming and expensive task. To this end, we propose an approach to CNN model selection using only unlabeled data. We develop this method based on a principle termed consistent relative confidence. Experimental results on benchmark datasets demonstrate the effectiveness and efficiency of our method.