A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization
This work addresses optimization challenges in machine learning and data analysis, offering a novel algorithmic framework for non-convex settings, though it appears incremental by building on existing ADMM and MM principles.
The authors tackled non-convex non-smooth optimization problems by proposing an inertial alternating direction method of multipliers (iADMM) framework, which unifies previous methods and demonstrates effectiveness on low-rank representation problems with proven convergence results.
In this paper, we propose an algorithmic framework, dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block of variables so as to not only unify the convergence analysis of previous ADMM that use specific surrogate functions in the MM step, but also lead to new efficient ADMM schemes. To the best of our knowledge, in the nonconvex nonsmooth setting, ADMM used in combination with the MM principle to update each block of variables, and ADMM combined with \emph{inertial terms for the primal variables} have not been studied in the literature. Under standard assumptions, we prove the subsequential convergence and global convergence for the generated sequence of iterates. We illustrate the effectiveness of iADMM on a class of nonconvex low-rank representation problems.