A Complete Analysis of the l_1,p Group-Lasso
This work provides a complete analysis and efficient algorithms for Group-Lasso variants, which is incremental but useful for researchers in machine learning regularization and multi-task learning.
The authors tackled the problem of characterizing solutions and developing efficient algorithms for the l_1,p Group-Lasso across all p-norms, showing that weak-coupling norms (p between 1.5 and 2) consistently outperform strong-coupling norms (p >> 2) in multi-task learning experiments on synthetic and real-world data.
The Group-Lasso is a well-known tool for joint regularization in machine learning methods. While the l_{1,2} and the l_{1,\infty} version have been studied in detail and efficient algorithms exist, there are still open questions regarding other l_{1,p} variants. We characterize conditions for solutions of the l_{1,p} Group-Lasso for all p-norms with 1 <= p <= \infty, and we present a unified active set algorithm. For all p-norms, a highly efficient projected gradient algorithm is presented. This new algorithm enables us to compare the prediction performance of many variants of the Group-Lasso in a multi-task learning setting, where the aim is to solve many learning problems in parallel which are coupled via the Group-Lasso constraint. We conduct large-scale experiments on synthetic data and on two real-world data sets. In accordance with theoretical characterizations of the different norms we observe that the weak-coupling norms with p between 1.5 and 2 consistently outperform the strong-coupling norms with p >> 2.