On Sparsity Inducing Regularization Methods for Machine Learning
This work provides a unified framework for sparsity regularization, which is incremental as it builds on existing methods like group Lasso and support vector machines.
The paper tackles the problem of solving sparsity regularization methods in machine learning by presenting a general approach for regularization problems where the regularizer is a convex function composed with a linear function, assuming the proximity operator is available, and applies it to methods like group Lasso and support vector machines.
During the past years there has been an explosion of interest in learning methods based on sparsity regularization. In this paper, we discuss a general class of such methods, in which the regularizer can be expressed as the composition of a convex function $ω$ with a linear function. This setting includes several methods such the group Lasso, the Fused Lasso, multi-task learning and many more. We present a general approach for solving regularization problems of this kind, under the assumption that the proximity operator of the function $ω$ is available. Furthermore, we comment on the application of this approach to support vector machines, a technique pioneered by the groundbreaking work of Vladimir Vapnik.