Robustness and Personalization in Federated Learning: A Unified Approach via Regularization
This work addresses robustness and personalization issues in federated learning for distributed systems, offering a unified approach that is incremental by building on existing methods.
The authors tackled the challenges of non-IID data, outliers, and stragglers in federated learning by introducing Fed+, a unified class of methods that provides robust aggregation and personalization, achieving convergence guarantees for convex and non-convex losses without statistical assumptions on data heterogeneity.
We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of methods is to better accommodate the real-world characteristics found in federated training, such as the lack of IID data across parties, the need for robustness to outliers or stragglers, and the requirement to perform well on party-specific datasets. We achieve this through a problem formulation that allows the central server to employ robust ways of aggregating the local models while keeping the structure of local computation intact. Without making any statistical assumption on the degree of heterogeneity of local data across parties, we provide convergence guarantees for Fed+ for convex and non-convex loss functions under different (robust) aggregation methods. The Fed+ theory is also equipped to handle heterogeneous computing environments including stragglers without additional assumptions; specifically, the convergence results cover the general setting where the number of local update steps across parties can vary. We demonstrate the benefits of Fed+ through extensive experiments across standard benchmark datasets.