Fast Generalized Matrix Regression with Applications in Machine Learning
This work addresses efficiency bottlenecks in matrix computations for machine learning applications, offering incremental improvements in speed and accuracy for specific tasks.
The paper tackles the generalized matrix regression problem by proposing a fast algorithm using sketching techniques, achieving a (1+ε) relative error with sketching sizes of order O(ε^{-1/2}) and demonstrating better performance in applications like symmetric positive definite matrix approximation and single-pass SVD compared to conventional methods.
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and stream singular value decomposition (SVD), etc. In this paper, we propose a fast generalized matrix regression algorithm (Fast GMR) which utilizes sketching technique to solve the GMR problem efficiently. Given error parameter $0<ε<1$, the Fast GMR algorithm can achieve a $(1+ε)$ relative error with the sketching sizes being of order $\cO(ε^{-1/2})$ for a large group of GMR problems. We apply the Fast GMR algorithm to the symmetric positive definite matrix approximation and single pass singular value decomposition and they achieve a better performance than conventional algorithms. Our empirical study also validates the effectiveness and efficiency of our proposed algorithms.