FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios
This addresses challenges in federated learning for scenarios with imbalanced data, though it appears incremental as it builds on momentum-based methods.
The paper tackles the problem of momentum-based federated learning struggling with non-IID and long-tailed data distributions, which cause biased models and convergence issues, and proposes FedWCM to dynamically adjust momentum, resolving non-convergence and outperforming existing methods.
Federated Learning (FL) enables decentralized model training while preserving data privacy. Despite its benefits, FL faces challenges with non-identically distributed (non-IID) data, especially in long-tailed scenarios with imbalanced class samples. Momentum-based FL methods, often used to accelerate FL convergence, struggle with these distributions, resulting in biased models and making FL hard to converge. To understand this challenge, we conduct extensive investigations into this phenomenon, accompanied by a layer-wise analysis of neural network behavior. Based on these insights, we propose FedWCM, a method that dynamically adjusts momentum using global and per-round data to correct directional biases introduced by long-tailed distributions. Extensive experiments show that FedWCM resolves non-convergence issues and outperforms existing methods, enhancing FL's efficiency and effectiveness in handling client heterogeneity and data imbalance.