On Regularized Sparse Logistic Regression
This work addresses a gap in sparse logistic regression methods for researchers and practitioners, but it is incremental as it builds on existing $\ell_1$-regularized approaches.
The paper tackles the problem of solving sparse logistic regression with nonconvex regularization, proposing a unified framework that extends to such terms and uses a line search for monotone convergence, resulting in effective classification and feature selection at lower computational cost.
Sparse logistic regression is for classification and feature selection simultaneously. Although many studies have been done to solve $\ell_1$-regularized logistic regression, there is no equivalently abundant work on solving sparse logistic regression with nonconvex regularization term. In this paper, we propose a unified framework to solve $\ell_1$-regularized logistic regression, which can be naturally extended to nonconvex regularization term, as long as certain requirement is satisfied. In addition, we also utilize a different line search criteria to guarantee monotone convergence for various regularization terms. Empirical experiments on binary classification tasks with real-world datasets demonstrate our proposed algorithms are capable of performing classification and feature selection effectively at a lower computational cost.