Analysis of regularized federated learning
This work addresses communication costs in federated learning for privacy-sensitive big data applications, but it is incremental as it builds on existing regularization methods.
The paper tackles the problem of communication efficiency in federated learning by improving the Loopless Local Gradient Descent algorithm with flexible step sizes and analyzing its convergence in non-convex and strongly convex settings, deriving convergence rates under specific conditions.
Federated learning is an efficient machine learning tool for dealing with heterogeneous big data and privacy protection. Federated learning methods with regularization can control the level of communications between the central and local machines. Stochastic gradient descent is often used for implementing such methods on heterogeneous big data, to reduce the communication costs. In this paper, we consider such an algorithm called Loopless Local Gradient Descent which has advantages in reducing the expected communications by controlling a probability level. We improve the method by allowing flexible step sizes and carry out novel analysis for the convergence of the algorithm in a non-convex setting in addition to the standard strongly convex setting. In the non-convex setting, we derive rates of convergence when the smooth objective function satisfies a Polyak-Łojasiewicz condition. When the objective function is strongly convex, a sufficient and necessary condition for the convergence in expectation is presented.