Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning
This addresses communication efficiency for edge devices in federated learning, representing an incremental improvement over existing sparsification methods.
The paper tackles communication bottlenecks in federated edge learning by proposing time-correlated sparsification with hybrid aggregation (TCS-H), which combines model compression and over-the-air computation to improve efficiency. Results show TCS-H achieves significantly higher accuracy than conventional top-K sparsification under limited communication resources, with both i.i.d. and non-i.i.d. data distributions.
Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.