A Field Guide to Federated Optimization
This work addresses the need for robust federated learning methods in practical applications, offering incremental guidance rather than introducing new algorithms.
The paper tackles the problem of designing and evaluating federated optimization algorithms for privacy-preserving distributed learning, providing practical guidelines and recommendations to improve real-world performance through effective simulations.
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection. The distributed learning process can be formulated as solving federated optimization problems, which emphasize communication efficiency, data heterogeneity, compatibility with privacy and system requirements, and other constraints that are not primary considerations in other problem settings. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms through concrete examples and practical implementation, with a focus on conducting effective simulations to infer real-world performance. The goal of this work is not to survey the current literature, but to inspire researchers and practitioners to design federated learning algorithms that can be used in various practical applications.