Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
This work addresses optimization challenges in image and signal processing by enhancing nonconvex methods, though it appears incremental as it builds on existing models like RPCA and LRR.
The authors tackled the problem of optimizing sparse and low-rank models like RPCA and LRR by proposing a novel nonconvex rank surrogate and a dual momentum trick in ADMM, resulting in improved convergence and smaller recovery errors, with experiments showing state-of-the-art performance in image completion, denoising, and clustering tasks.
In this manuscript, we research on the behaviors of surrogates for the rank function on different image processing problems and their optimization algorithms. We first propose a novel nonconvex rank surrogate on the general rank minimization problem and apply this to the corrupted image completion problem. Then, we propose that nonconvex rank surrogates can be introduced into two well-known sparse and low-rank models: Robust Principal Component Analysis (RPCA) and Low-Rank Representation (LRR). For optimization, we use alternating direction method of multipliers (ADMM) and propose a trick, which is called the dual momentum. We add the difference of the dual variable between the current and the last iteration with a weight. This trick can avoid the local minimum problem and make the algorithm converge to a solution with smaller recovery error in the nonconvex optimization problem. Also, it can boost the convergence when the variable updates too slowly. We also give a severe proof and verify that the proposed algorithms are convergent. Then, several experiments are conducted, including image completion, denoising, and spectral clustering with outlier detection. These experiments show that the proposed methods are effective in image and signal processing applications, and have the best performance compared with state-of-the-art methods.