Efficient Wireless Federated Learning via Low-Rank Gradient Factorization
This work addresses communication bottlenecks in federated learning for wireless systems, offering incremental improvements over existing methods.
The paper tackles the problem of high communication costs in wireless federated learning by proposing a low-rank gradient compression method, which reduces total communication costs by at least 33% while maintaining the same inference performance on the Cifar-10 dataset.
This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes.