Sharp Gaussian approximations for Decentralized Federated Learning
This work provides theoretical tools for improving robustness and inference in privacy-sensitive collaborative learning, though it is incremental as it builds on existing local SGD methods.
The paper tackles the lack of statistical guarantees beyond convergence in decentralized federated learning by proving Gaussian approximations for local SGD, enabling bootstrap procedures for inference and adversarial attack detection, with extensive simulations supporting the results.
Federated Learning has gained traction in privacy-sensitive collaborative environments, with local SGD emerging as a key optimization method in decentralized settings. While its convergence properties are well-studied, asymptotic statistical guarantees beyond convergence remain limited. In this paper, we present two generalized Gaussian approximation results for local SGD and explore their implications. First, we prove a Berry-Esseen theorem for the final local SGD iterates, enabling valid multiplier bootstrap procedures. Second, motivated by robustness considerations, we introduce two distinct time-uniform Gaussian approximations for the entire trajectory of local SGD. The time-uniform approximations support Gaussian bootstrap-based tests for detecting adversarial attacks. Extensive simulations are provided to support our theoretical results.