Safe Screening for Logistic Regression with $\ell_0$-$\ell_2$ Regularization
This work addresses computational efficiency for logistic regression in high-dimensional settings, but it is incremental as it builds on existing screening and regularization methods.
The paper tackles the problem of speeding up logistic regression with ℓ0-ℓ2 regularization by developing safe screening rules that remove features before solving, based on Fenchel dual lower bounds from strong conic relaxations, resulting in substantial computational speed-ups as shown in numerical experiments with real and synthetic data.
In logistic regression, it is often desirable to utilize regularization to promote sparse solutions, particularly for problems with a large number of features compared to available labels. In this paper, we present screening rules that safely remove features from logistic regression with $\ell_0-\ell_2$ regularization before solving the problem. The proposed safe screening rules are based on lower bounds from the Fenchel dual of strong conic relaxations of the logistic regression problem. Numerical experiments with real and synthetic data suggest that a high percentage of the features can be effectively and safely removed apriori, leading to substantial speed-up in the computations.