Improvement of Batch Normalization in Imbalanced Data
This addresses a practical issue in machine learning for applications with skewed data distributions, but it is incremental as it builds on existing methods.
The paper tackled the problem of neural network classification with imbalanced data by proposing a modified batch normalization to correct a size-mismatch issue when combined with weighted loss functions, demonstrating its effectiveness in such environments.
In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective method for imbalanced data sets. We consider a combination of weighted loss function and batch normalization (BN) in this study. BN is a powerful standard technique in the recent developments in deep learning. A simple combination of both methods leads to a size-mismatch problem due to a mismatch between interpretations of effective size of data set in both methods. We propose a simple modification to BN to correct the size-mismatch and demonstrate that this modified BN is effective in data-imbalanced environment.