Improving Sketching Algorithms for Low-Rank Matrix Approximation via Sketch-Power Iterations
This work addresses the need for efficient low-rank approximation algorithms in streaming or memory-limited environments, representing an incremental improvement over existing one-pass methods.
The paper tackles the problem of improving low-rank matrix approximation in streaming or memory-constrained settings by introducing sketch-power iterations, which enable power-like iterations with only a single data pass, and shows that applying one or two iterations can substantially improve approximation accuracy under the same storage constraints.
Power iteration can improve the accuracy of randomized SVD, but requires multiple data passes, making it impractical in streaming or memory-constrained settings. We introduce a lightweight yet effective sketch-power iteration, allowing power-like iterations with only a single pass of the data, which can be incorporated into one-pass algorithms for low-rank approximation. As an example, we integrate the sketch-power iteration into a one-pass algorithm proposed by Tropp et al., and introduce strategies to reduce its storage cost. We establish meaningful error bounds: given a fixed storage budget, the sketch sizes derived from the bounds closely match the optimal ones observed in reality. This allows one to preselect reasonable parameters. Numerical experiments on both synthetic and real-world datasets indicate that, under the same storage constraints, applying one or two sketch-power iterations can substantially improve the approximation accuracy of the considered one-pass algorithms. In particular, experiments on real data with flat spectrum show that the method can approximate the dominant singular vectors well.