On the Global Convergence of Majorization Minimization Algorithms for Nonconvex Optimization Problems
Provides theoretical convergence guarantees for a widely used class of optimization algorithms in machine learning, addressing a known gap for nonconvex problems.
The paper proves global convergence of majorization-minimization (MM) algorithms for nonconvex optimization problems by leveraging the Kurdyka-Łojasiewicz inequality, extending the result to the concave-convex procedure (CCCP).
In this paper, we study the global convergence of majorization minimization (MM) algorithms for solving nonconvex regularized optimization problems. MM algorithms have received great attention in machine learning. However, when applied to nonconvex optimization problems, the convergence of MM algorithms is a challenging issue. We introduce theory of the Kurdyka- Lojasiewicz inequality to address this issue. In particular, we show that many nonconvex problems enjoy the Kurdyka- Lojasiewicz property and establish the global convergence result of the corresponding MM procedure. We also extend our result to a well known method that called CCCP (concave-convex procedure).