Towards Heterogeneous Clients with Elastic Federated Learning
This addresses bias issues in federated learning for heterogeneous networks like edge devices, but it is incremental as it builds on existing methods to improve handling of non-IID data and low participation.
The paper tackles the problem of bias in federated learning due to non-IID data and low client participation by proposing Elastic Federated Learning (EFL), an unbiased algorithm that reduces parameter volatility and uses incomplete updates, with theoretical convergence guarantees and competitive empirical performance in robustness and efficiency.
Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses, while keeping the data local. Training in heterogeneous and potentially massive networks introduces bias into the system, which is originated from the non-IID data and the low participation rate in reality. In this paper, we propose Elastic Federated Learning (EFL), an unbiased algorithm to tackle the heterogeneity in the system, which makes the most informative parameters less volatile during training, and utilizes the incomplete local updates. It is an efficient and effective algorithm that compresses both upstream and downstream communications. Theoretically, the algorithm has convergence guarantee when training on the non-IID data at the low participation rate. Empirical experiments corroborate the competitive performance of EFL framework on the robustness and the efficiency.