Achieving Linear Speedup for Composite Federated Learning
It addresses efficiency and scalability issues in federated learning for distributed systems, though it appears incremental as it builds on existing methods with specific improvements.
The paper tackles the problem of composite federated learning with nonconvex losses and nonsmooth regularizers by proposing FedNMap, which achieves linear speedup in both the number of clients and local updates, the first such result for this setting.
This paper proposes FedNMap, a normal map-based method for composite federated learning, where the objective consists of a smooth loss and a possibly nonsmooth regularizer. FedNMap leverages a normal map-based update scheme to handle the nonsmooth term and incorporates a local correction strategy to mitigate the impact of data heterogeneity across clients. Under standard assumptions, including smooth local losses, weak convexity of the regularizer, and bounded stochastic gradient variance, FedNMap achieves linear speedup with respect to both the number of clients $n$ and the number of local updates $Q$ for nonconvex losses, both with and without the Polyak-Łojasiewicz (PL) condition. To our knowledge, this is the first result establishing linear speedup for nonconvex composite federated learning.