A dual Newton based preconditioned proximal point algorithm for exclusive lasso models
This work provides an incremental improvement in optimization methods for group sparsity models, benefiting researchers and practitioners in machine learning.
The authors tackled the problem of efficiently solving machine learning models with exclusive lasso regularization by proposing a dual Newton based preconditioned proximal point algorithm (PPDNA), which demonstrated superior performance in numerical experiments compared to other state-of-the-art algorithms.
The exclusive lasso (also known as elitist lasso) regularization has become popular recently due to its superior performance on group sparsity. Compared to the group lasso regularization which enforces the competition on variables among different groups, the exclusive lasso regularization also enforces the competition within each group. In this paper, we propose a highly efficient dual Newton based preconditioned proximal point algorithm (PPDNA) to solve machine learning models involving the exclusive lasso regularizer. As an important ingredient, we provide a rigorous proof for deriving the closed-form solution to the proximal mapping of the weighted exclusive lasso regularizer. In addition, we derive the corresponding HS-Jacobian to the proximal mapping and analyze its structure --- which plays an essential role in the efficient computation of the PPA subproblem via applying a semismooth Newton method on its dual. Various numerical experiments in this paper demonstrate the superior performance of the proposed PPDNA against other state-of-the-art numerical algorithms.