An Analytical Model for Overparameterized Learning Under Class Imbalance
This work addresses class imbalance in machine learning, providing analytical insights for practitioners, but it is incremental as it builds on existing methods with theoretical analysis.
The authors tackled class-imbalanced linear classification in high-dimensional Gaussian mixtures by developing a closed-form approximation for test error of methods like logit adjustment, enabling analytical tuning and comparison to overcome standard cross-entropy pitfalls, with validation on simulated and imbalanced datasets such as CIFAR10.
We study class-imbalanced linear classification in a high-dimensional Gaussian mixture model. We develop a tight, closed form approximation for the test error of several practical learning methods, including logit adjustment and class dependent temperature. Our approximation allows us to analytically tune and compare these methods, highlighting how and when they overcome the pitfalls of standard cross-entropy minimization. We test our theoretical findings on simulated data and imbalanced CIFAR10, MNIST and FashionMNIST datasets.