A Support Detection and Root Finding Approach for Learning High-dimensional Generalized Linear Models
This work addresses feature selection for high-dimensional data analysis, offering incremental improvements over existing methods like Lasso and MCP.
The paper tackles feature selection in high-dimensional sparse generalized linear models by proposing a support detection and root finding method (GSDAR), which achieves exponential error decay to optimal order and can recover the oracle estimator under strong signal conditions.
Feature selection is important for modeling high-dimensional data, where the number of variables can be much larger than the sample size. In this paper, we develop a support detection and root finding procedure to learn the high dimensional sparse generalized linear models and denote this method by GSDAR. Based on the KKT condition for $\ell_0$-penalized maximum likelihood estimations, GSDAR generates a sequence of estimators iteratively. Under some restricted invertibility conditions on the maximum likelihood function and sparsity assumption on the target coefficients, the errors of the proposed estimate decays exponentially to the optimal order. Moreover, the oracle estimator can be recovered if the target signal is stronger than the detectable level. We conduct simulations and real data analysis to illustrate the advantages of our proposed method over several existing methods, including Lasso and MCP.