Mitigating Participation Imbalance Bias in Asynchronous Federated Learning
This work addresses a critical bias issue in federated learning for distributed systems, offering incremental improvements to existing AFL frameworks.
The paper tackles the problem of heterogeneity amplification in Asynchronous Federated Learning (AFL), where faster clients bias the global model due to non-IID data and update delays, by proposing ACE and ACED methods that mitigate participation imbalance and demonstrate robust performance across diverse settings.
In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client's contribution. As a result, clients perform their local training on different model versions, causing information staleness (delay). In federated environments with non-IID local data distributions, this asynchronous pattern amplifies the adverse effect of client heterogeneity (due to different data distribution, local objectives, etc.), as faster clients contribute more frequent updates, biasing the global model. We term this phenomenon heterogeneity amplification. Our work provides a theoretical analysis that maps AFL design choices to their resulting error sources when heterogeneity amplification occurs. Guided by our analysis, we propose ACE (All-Client Engagement AFL), which mitigates participation imbalance through immediate, non-buffered updates that use the latest information available from all clients. We also introduce a delay-aware variant, ACED, to balance client diversity against update staleness. Experiments on different models for different tasks across diverse heterogeneity and delay settings validate our analysis and demonstrate the robust performance of our approaches.