MAGMA: Multi-level accelerated gradient mirror descent algorithm for large-scale convex composite minimization
This work addresses efficiency issues in large-scale convex optimization for applications like image processing and inverse problems, offering a faster algorithm with optimal theoretical guarantees.
The paper tackles the slow convergence of accelerated first-order methods for large-scale convex composite minimization by proposing a multi-level accelerated gradient mirror descent algorithm that exploits varying fidelity models, achieving an optimal O(1/√ε) convergence rate and demonstrating several times faster performance on large-scale face recognition problems.
Composite convex optimization models arise in several applications, and are especially prevalent in inverse problems with a sparsity inducing norm and in general convex optimization with simple constraints. The most widely used algorithms for convex composite models are accelerated first order methods, however they can take a large number of iterations to compute an acceptable solution for large-scale problems. In this paper we propose to speed up first order methods by taking advantage of the structure present in many applications and in image processing in particular. Our method is based on multi-level optimization methods and exploits the fact that many applications that give rise to large scale models can be modelled using varying degrees of fidelity. We use Nesterov's acceleration techniques together with the multi-level approach to achieve $\mathcal{O}(1/\sqrtε)$ convergence rate, where $ε$ denotes the desired accuracy. The proposed method has a better convergence rate than any other existing multi-level method for convex problems, and in addition has the same rate as accelerated methods, which is known to be optimal for first-order methods. Moreover, as our numerical experiments show, on large-scale face recognition problems our algorithm is several times faster than the state of the art.