An Optimization Framework with Flexible Inexact Inner Iterations for Nonconvex and Nonsmooth Programming
This work addresses the need for reliable optimization methods in machine learning and computer vision, offering a flexible framework that can incorporate various numerical algorithms while ensuring convergence, though it is incremental as it builds on existing ideas of inexact inner iterations.
The paper tackles the challenge of designing fast and flexible optimization schemes with theoretical guarantees for general nonconvex and nonsmooth programming (NNPs), common in vision and learning tasks, by proposing the inexact proximal alternating direction method (IPAD) framework, which demonstrates superiority and flexibility in numerical experiments on synthesized and real-world data.
In recent years, numerous vision and learning tasks have been (re)formulated as nonconvex and nonsmooth programmings(NNPs). Although some algorithms have been proposed for particular problems, designing fast and flexible optimization schemes with theoretical guarantee is a challenging task for general NNPs. It has been investigated that performing inexact inner iterations often benefit to special applications case by case, but their convergence behaviors are still unclear. Motivated by these practical experiences, this paper designs a novel algorithmic framework, named inexact proximal alternating direction method (IPAD) for solving general NNPs. We demonstrate that any numerical algorithms can be incorporated into IPAD for solving subproblems and the convergence of the resulting hybrid schemes can be consistently guaranteed by a series of simple error conditions. Beyond the guarantee in theory, numerical experiments on both synthesized and real-world data further demonstrate the superiority and flexibility of our IPAD framework for practical use.