Learning Imbalanced Datasets with Maximum Margin Loss
This work addresses the challenge of underfitting for minority classes in imbalanced datasets, which is a common issue in machine learning applications, but it is incremental as it builds on existing label-distribution-aware margin (LDAM) loss strategies.
The paper tackles the problem of class imbalance in datasets by proposing a Maximum Margin (MM) loss function to improve generalization for minority classes, achieving competitive results on artificially imbalanced CIFAR-10/100 benchmarks.
A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the trained model tends to predict the majority of classes rather than the minority ones. That is, underfitting for minority classes seems to be one of the challenges of generalization. For a good generalization of the minority classes, we design a new Maximum Margin (MM) loss function, motivated by minimizing a margin-based generalization bound through the shifting decision bound. The theoretically-principled label-distribution-aware margin (LDAM) loss was successfully applied with prior strategies such as re-weighting or re-sampling along with the effective training schedule. However, they did not investigate the maximum margin loss function yet. In this study, we investigate the performances of two types of hard maximum margin-based decision boundary shift with LDAM's training schedule on artificially imbalanced CIFAR-10/100 for fair comparisons and effectiveness.