Sequential Least-Squares Estimators with Fast Randomized Sketching for Linear Statistical Models
This work addresses computational efficiency in large-scale linear modeling, offering an incremental improvement for researchers and practitioners in statistics and machine learning.
The paper tackles the estimation problem for large-scale linear statistical models by proposing SLSE-FRS, a method that integrates Sketch-and-Solve and Iterative-Sketching to iteratively refine estimators, and it outperforms state-of-the-art methods like PCG and IDS in numerical experiments.
We propose a novel randomized framework for the estimation problem of large-scale linear statistical models, namely Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), which integrates Sketch-and-Solve and Iterative-Sketching methods for the first time. By iteratively constructing and solving sketched least-squares (LS) subproblems with increasing sketch sizes to achieve better precisions, SLSE-FRS gradually refines the estimators of the true parameter vector, ultimately producing high-precision estimators. We analyze the convergence properties of SLSE-FRS, and provide its efficient implementation. Numerical experiments show that SLSE-FRS outperforms the state-of-the-art methods, namely the Preconditioned Conjugate Gradient (PCG) method, and the Iterative Double Sketching (IDS) method.