Weighted Orthogonal Components Regression Analysis
This work addresses predictive modeling in statistics and machine learning, offering incremental improvements by generalizing existing regression methods.
The authors tackled the problem of improving predictive performance in multiple linear regression by proposing a general framework called weighted orthogonal components regression (WOCR), which includes methods like ridge regression as special cases and uses monotonicity in orthogonal components for parameterization, resulting in enhanced predictive performance as demonstrated through simulated studies and real data examples.
In the multiple linear regression setting, we propose a general framework, termed weighted orthogonal components regression (WOCR), which encompasses many known methods as special cases, including ridge regression and principal components regression. WOCR makes use of the monotonicity inherent in orthogonal components to parameterize the weight function. The formulation allows for efficient determination of tuning parameters and hence is computationally advantageous. Moreover, WOCR offers insights for deriving new better variants. Specifically, we advocate weighting components based on their correlations with the response, which leads to enhanced predictive performance. Both simulated studies and real data examples are provided to assess and illustrate the advantages of the proposed methods.