Dynamic Sasvi: Strong Safe Screening for Norm-Regularized Least Squares
This work provides an incremental improvement in computational efficiency for researchers and practitioners working with sparse optimization problems, specifically norm-regularized least squares.
This paper introduces a strong safe screening rule called "dynamic Sasvi" for norm-regularized least squares problems. It improves upon existing methods by not requiring an exact solution to a more strongly regularized problem, leading to the elimination of more features and increased solver speed both theoretically and experimentally.
A recently introduced technique for a sparse optimization problem called "safe screening" allows us to identify irrelevant variables in the early stage of optimization. In this paper, we first propose a flexible framework for safe screening based on the Fenchel-Rockafellar duality and then derive a strong safe screening rule for norm-regularized least squares by the framework. We call the proposed screening rule for norm-regularized least squares "dynamic Sasvi" because it can be interpreted as a generalization of Sasvi. Unlike the original Sasvi, it does not require the exact solution of a more strongly regularized problem; hence, it works safely in practice. We show that our screening rule can eliminate more features and increase the speed of the solver in comparison with other screening rules both theoretically and experimentally.