Posterior Re-calibration for Imbalanced Datasets
This addresses the issue of label imbalance and distribution shift in classification and segmentation tasks, providing a unified solution that is incremental by combining with existing methods.
The paper tackles the problem of neural networks performing poorly under imbalanced training label distributions and shifts in testing label distributions by deriving a post-training prior rebalancing technique from an optimal Bayes classifier perspective, achieving state-of-the-art accuracy on six datasets including large-scale ones like iNaturalist and Synthia.
Neural Networks can perform poorly when the training label distribution is heavily imbalanced, as well as when the testing data differs from the training distribution. In order to deal with shift in the testing label distribution, which imbalance causes, we motivate the problem from the perspective of an optimal Bayes classifier and derive a post-training prior rebalancing technique that can be solved through a KL-divergence based optimization. This method allows a flexible post-training hyper-parameter to be efficiently tuned on a validation set and effectively modify the classifier margin to deal with this imbalance. We further combine this method with existing likelihood shift methods, re-interpreting them from the same Bayesian perspective, and demonstrating that our method can deal with both problems in a unified way. The resulting algorithm can be conveniently used on probabilistic classification problems agnostic to underlying architectures. Our results on six different datasets and five different architectures show state of art accuracy, including on large-scale imbalanced datasets such as iNaturalist for classification and Synthia for semantic segmentation. Please see https://github.com/GT-RIPL/UNO-IC.git for implementation.