The Hitchhiker's Guide to Prior-Shift Adaptation
This addresses prior shift adaptation for computer vision classifiers, offering an incremental improvement over existing methods.
The paper tackles the problem of prior shift in computer vision classification, where test-time class priors differ from training priors, by proposing a novel method to address inconsistencies in prior estimation that lead to negative values. Experiments show the method achieves state-of-the-art results, increasing recognition accuracy by 1.1% and 3.4% in two tasks with imbalanced priors.
In many computer vision classification tasks, class priors at test time often differ from priors on the training set. In the case of such prior shift, classifiers must be adapted correspondingly to maintain close to optimal performance. This paper analyzes methods for adaptation of probabilistic classifiers to new priors and for estimating new priors on an unlabeled test set. We propose a novel method to address a known issue of prior estimation methods based on confusion matrices, where inconsistent estimates of decision probabilities and confusion matrices lead to negative values in the estimated priors. Experiments on fine-grained image classification datasets provide insight into the best practice of prior shift estimation and classifier adaptation, and show that the proposed method achieves state-of-the-art results in prior adaptation. Applying the best practice to two tasks with naturally imbalanced priors, learning from web-crawled images and plant species classification, increased the recognition accuracy by 1.1% and 3.4% respectively.