Robust and Sparse Regression in GLM by Stochastic Optimization
This work addresses robustness and efficiency in high-dimensional regression for statistical modeling, but it is incremental as it extends existing methods to a broader class of models.
The paper tackles the problem of making sparse generalized linear models robust against outliers by extending robust and sparse linear regression based on γ-divergence to GLMs using stochastic optimization, and it shows that the proposed method outperforms comparative methods in numerical experiments and real data analysis.
The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regression based on the $γ$-divergence. The estimator of the $γ$-divergence has strong robustness under heavy contamination. In this paper, we extend the robust and sparse linear regression based on the $γ$-divergence to the robust and sparse GLM based on the $γ$-divergence with a stochastic optimization approach in order to obtain the estimate. We adopt the randomized stochastic projected gradient descent as a stochastic optimization approach and extend the established convergence property to the classical first-order necessary condition. By virtue of the stochastic optimization approach, we can efficiently estimate parameters for very large problems. Particularly, we show the linear regression, logistic regression and Poisson regression with $L_1$ regularization in detail as specific examples of robust and sparse GLM. In numerical experiments and real data analysis, the proposed method outperformed comparative methods.