An Experimental Study of Class Imbalance in Federated Learning
This addresses performance degradation in federated learning systems due to class imbalance, which is an incremental contribution as it builds on existing federated learning methods by focusing on a specific bottleneck.
The paper tackles the problem of class imbalance in federated learning by proposing new metrics to define it and analyzing its impact on global model performance. Results show that higher global imbalance and larger local differences degrade performance and slow convergence, with specific metrics like MID and WCS quantifying these effects.
Federated learning is a distributed machine learning paradigm that trains a global model for prediction based on a number of local models at clients while local data privacy is preserved. Class imbalance is believed to be one of the factors that degrades the global model performance. However, there has been very little research on if and how class imbalance can affect the global performance. class imbalance in federated learning is much more complex than that in traditional non-distributed machine learning, due to different class imbalance situations at local clients. Class imbalance needs to be re-defined in distributed learning environments. In this paper, first, we propose two new metrics to define class imbalance -- the global class imbalance degree (MID) and the local difference of class imbalance among clients (WCS). Then, we conduct extensive experiments to analyze the impact of class imbalance on the global performance in various scenarios based on our definition. Our results show that a higher MID and a larger WCS degrade more the performance of the global model. Besides, WCS is shown to slow down the convergence of the global model by misdirecting the optimization.