Composite federated learning with heterogeneous data
This addresses the challenge of efficient and robust federated learning for distributed systems with non-i.i.d. data, though it appears incremental as it builds on existing FL methods.
The paper tackles the composite federated learning problem with non-smooth regularization and heterogeneous data, proposing an algorithm that decouples proximal operators and communication to reduce client drift and communication frequency, achieving linear convergence and outperforming state-of-the-art methods in experiments.
We propose a novel algorithm for solving the composite Federated Learning (FL) problem. This algorithm manages non-smooth regularization by strategically decoupling the proximal operator and communication, and addresses client drift without any assumptions about data similarity. Moreover, each worker uses local updates to reduce the communication frequency with the server and transmits only a $d$-dimensional vector per communication round. We prove that our algorithm converges linearly to a neighborhood of the optimal solution and demonstrate the superiority of our algorithm over state-of-the-art methods in numerical experiments.