Composite Self-Concordant Minimization
This provides a theoretical and algorithmic advancement for optimization in machine learning, though it appears incremental as it builds on existing composite minimization methods.
The authors tackled the problem of minimizing composite functions with self-concordant smooth parts and non-smooth convex parts, developing a variable metric framework with analytic step-size procedures that converges without Lipschitz gradient assumptions. They demonstrated this framework numerically on synthetic and real data.
We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator. We theoretically establish the convergence of our framework without relying on the usual Lipschitz gradient assumption on the smooth part. An important highlight of our work is a new set of analytic step-size selection and correction procedures based on the structure of the problem. We describe concrete algorithmic instances of our framework for several interesting applications and demonstrate them numerically on both synthetic and real data.