Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping
This addresses robustness issues in federated edge learning systems for applications like IoT, though it is incremental as it builds on existing gradient clipping methods.
The paper tackles the problem of heavy-tailed noise degrading training performance in over-the-air federated learning by proposing a novel gradient clipping method called Median Anchored Clipping (MAC), which effectively mitigates this noise and enhances system robustness as shown in experiments.
Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness.