Proximal Newton-type methods for minimizing composite functions
This provides a unified theoretical framework for optimization methods in domains like bioinformatics and machine learning, though it appears incremental as it generalizes existing approaches.
The authors tackled the problem of minimizing composite functions (sum of smooth and nonsmooth convex functions) by generalizing Newton-type methods to handle nonsmooth components via proximal mappings. They showed these proximal Newton-type methods maintain convergence properties similar to smooth Newton methods, even with inexact search directions, and demonstrated that many existing methods in bioinformatics, signal processing, and statistical learning are special cases of their framework.
We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing smooth functions, even when search directions are computed inexactly. Many popular methods tailored to problems arising in bioinformatics, signal processing, and statistical learning are special cases of proximal Newton-type methods, and our analysis yields new convergence results for some of these methods.