Sparse principal component regression via singular value decomposition approach
This is an incremental improvement for statistical modeling and data analysis, addressing a specific limitation in PCR methods.
The authors tackled the problem of principal component regression (PCR) lacking response variable information by proposing a one-stage procedure that integrates regression and PCA losses with sparse regularization, resulting in principal component loadings that incorporate both explanatory and response variable information.
Principal component regression (PCR) is a two-stage procedure: the first stage performs principal component analysis (PCA) and the second stage constructs a regression model whose explanatory variables are replaced by principal components obtained by the first stage. Since PCA is performed by using only explanatory variables, the principal components have no information about the response variable. To address the problem, we propose a one-stage procedure for PCR in terms of singular value decomposition approach. Our approach is based upon two loss functions, a regression loss and a PCA loss, with sparse regularization. The proposed method enables us to obtain principal component loadings that possess information about both explanatory variables and a response variable. An estimation algorithm is developed by using alternating direction method of multipliers. We conduct numerical studies to show the effectiveness of the proposed method.