SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization
This addresses optimization challenges in machine learning for researchers and practitioners, though it appears incremental as it builds on existing proximal and gradient methods.
The paper tackles nonconvex regularized finite sum minimization problems by introducing SPIRAL, an incremental proximal algorithm that achieves superlinear convergence under mild assumptions, with simulation results showing it is competitive to state-of-the-art methods on various problem types.
We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. Each iteration of SPIRAL consists of an inner and an outer loop. It combines incremental gradient updates with a linesearch that has the remarkable property of never being triggered asymptotically, leading to superlinear convergence under mild assumptions at the limit point. Simulation results with L-BFGS directions on different convex, nonconvex, and non-Lipschitz differentiable problems show that our algorithm, as well as its adaptive variant, are competitive to the state of the art.