Global Convergence of Model Function Based Bregman Proximal Minimization Algorithms
This work provides a more general theoretical framework for designing optimization algorithms for a wider range of nonconvex nonsmooth problems, which is significant for researchers and practitioners dealing with complex optimization tasks.
The paper addresses the limitation of the L-smad property in handling nonsmooth functions in optimization by introducing the MAP property, which is applicable to a broader class of nonconvex nonsmooth composite problems. Based on this, they propose Model BPG, a globally convergent algorithm, demonstrating superior performance on phase retrieval and Poisson linear inverse problems compared to a state-of-the-art method.
Lipschitz continuity of the gradient mapping of a continuously differentiable function plays a crucial role in designing various optimization algorithms. However, many functions arising in practical applications such as low rank matrix factorization or deep neural network problems do not have a Lipschitz continuous gradient. This led to the development of a generalized notion known as the $L$-smad property, which is based on generalized proximity measures called Bregman distances. However, the $L$-smad property cannot handle nonsmooth functions, for example, simple nonsmooth functions like $\abs{x^4-1}$ and also many practical composite problems are out of scope. We fix this issue by proposing the MAP property, which generalizes the $L$-smad property and is also valid for a large class of nonconvex nonsmooth composite problems. Based on the proposed MAP property, we propose a globally convergent algorithm called Model BPG, that unifies several existing algorithms. The convergence analysis is based on a new Lyapunov function. We also numerically illustrate the superior performance of Model BPG on standard phase retrieval problems, robust phase retrieval problems, and Poisson linear inverse problems, when compared to a state of the art optimization method that is valid for generic nonconvex nonsmooth optimization problems.