First Analysis of Local GD on Heterogeneous Data
This addresses the problem of federated learning with heterogeneous data, but it is incremental as it focuses on theoretical analysis without new empirical results.
The paper provides the first convergence analysis of local gradient descent for minimizing the average of smooth and convex functions, showing that in a low accuracy regime, it has the same communication complexity as gradient descent.
We provide the first convergence analysis of local gradient descent for minimizing the average of smooth and convex but otherwise arbitrary functions. Problems of this form and local gradient descent as a solution method are of importance in federated learning, where each function is based on private data stored by a user on a mobile device, and the data of different users can be arbitrarily heterogeneous. We show that in a low accuracy regime, the method has the same communication complexity as gradient descent.