Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent
This work addresses computational bottlenecks in large-scale regression for data scientists, though it appears incremental as it builds on known techniques like Ridge Regression and Principal Component Analysis.
The authors tackled large-scale regression by proposing LING, a two-stage algorithm that matches Ridge Regression's risk while being significantly faster, achieving competitive prediction accuracy and computational efficiency compared to existing methods like Gradient Descent and Principal Component Regression on simulated and real datasets.
We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regression under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency compared with other large scale regression algorithms like Gradient Descent, Stochastic Gradient Descent and Principal Component Regression on both simulated and real datasets.