Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity
It addresses security and efficiency challenges in federated learning for distributed systems, offering a resilient method against malicious attacks, though it is incremental with theoretical extensions.
This paper tackles federated learning under Byzantine attacks and data heterogeneity by proposing the Robust Average Gradient Algorithm (RAGA), which achieves convergence rates of O(1/T^{2/3-δ}) for non-convex functions and linear for strongly-convex functions, with experimental validation showing robustness and superior performance.
This paper addresses federated learning (FL) in the context of malicious Byzantine attacks and data heterogeneity. We introduce a novel Robust Average Gradient Algorithm (RAGA), which uses the geometric median for aggregation and {allows flexible round number for local updates.} Unlike most existing resilient approaches, which base their convergence analysis on strongly-convex loss functions or homogeneously distributed datasets, this work conducts convergence analysis for both strongly-convex and non-convex loss functions over heterogeneous datasets. The theoretical analysis indicates that as long as the fraction of the {data} from malicious users is less than half, RAGA can achieve convergence at a rate of $\mathcal{O}({1}/{T^{2/3- δ}})$ for non-convex loss functions, where $T$ is the iteration number and $δ\in (0, 2/3)$. For strongly-convex loss functions, the convergence rate is linear. Furthermore, the stationary point or global optimal solution is shown to be attainable as data heterogeneity diminishes. Experimental results validate the robustness of RAGA against Byzantine attacks and demonstrate its superior convergence performance compared to baselines under varying intensities of Byzantine attacks on heterogeneous datasets.