Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting
This addresses the problem of securing federated learning against attacks for distributed systems, but it is incremental as it builds on existing robust aggregation rules.
The paper tackles the vulnerability of federated learning to Byzantine attacks in non-IID data settings by proposing GAS, a gradient splitting approach that adapts existing robust aggregation rules, achieving improved convergence and performance as verified on real-world datasets.
Federated learning has exhibited vulnerabilities to Byzantine attacks, where the Byzantine attackers can send arbitrary gradients to a central server to destroy the convergence and performance of the global model. A wealth of robust AGgregation Rules (AGRs) have been proposed to defend against Byzantine attacks. However, Byzantine clients can still circumvent robust AGRs when data is non-Identically and Independently Distributed (non-IID). In this paper, we first reveal the root causes of performance degradation of current robust AGRs in non-IID settings: the curse of dimensionality and gradient heterogeneity. In order to address this issue, we propose GAS, a \shorten approach that can successfully adapt existing robust AGRs to non-IID settings. We also provide a detailed convergence analysis when the existing robust AGRs are combined with GAS. Experiments on various real-world datasets verify the efficacy of our proposed GAS. The implementation code is provided in https://github.com/YuchenLiu-a/byzantine-gas.