On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems
This addresses federated bilevel optimization for distributed machine learning applications, representing an incremental improvement with a novel convergence guarantee.
The paper tackled federated bilevel optimization by developing two momentum-based algorithms, achieving a convergence rate with linear speedup in the number of devices and providing sample and communication complexities, as confirmed by experimental results.
In this paper, we studied the federated bilevel optimization problem, which has widespread applications in machine learning. In particular, we developed two momentum-based algorithms for optimizing this kind of problem and established the convergence rate of our two algorithms, providing the sample and communication complexities. Importantly, to the best of our knowledge, our convergence rate is the first one achieving the linear speedup with respect to the number of devices for federated bilevel optimization algorithms. At last, our extensive experimental results confirm the effectiveness of our two algorithms.