Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
This work addresses performance degradation in wireless federated learning due to data heterogeneity, which is an incremental improvement for applications in wireless networks with distributed model training.
The paper tackled the challenge of data heterogeneity in wireless federated learning by developing a closed-form upper bound for the loss function and jointly optimizing client scheduling, resource allocation, and local training epochs, resulting in improved learning accuracy and reduced energy consumption in experiments on real-world datasets.
With the rapid proliferation of smart mobile devices, federated learning (FL) has been widely considered for application in wireless networks for distributed model training. However, data heterogeneity, e.g., non-independently identically distributions and different sizes of training data among clients, poses major challenges to wireless FL. Limited communication resources complicate the implementation of fair scheduling which is required for training on heterogeneous data, and further deteriorate the overall performance. To address this issue, this paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. Specifically, we first develop a closed-form expression for an upper bound on the FL loss function, with a particular emphasis on data heterogeneity described by a dataset size vector and a data divergence vector. Then we formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE). Next, via the Lyapunov drift technique, we transform the CRE optimization problem into a series of tractable problems. Extensive experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.