Accurate and fast matrix factorization for low-rank learning
This work addresses computational bottlenecks in low-rank learning for applications requiring high accuracy in singular values and vectors, such as image similarity tasks, but it is incremental as it builds on existing Krylov subspace techniques.
The paper tackles partial singular value decomposition and numerical rank estimation for huge matrices, proposing methods based on Krylov subspaces and Ritz vectors that show advantages in speed and accuracy over traditional and randomized SVD methods, as demonstrated in experiments on image datasets like MNIST and USPS.
In this paper, we tackle two important problems in low-rank learning, which are partial singular value decomposition and numerical rank estimation of huge matrices. By using the concepts of Krylov subspaces such as Golub-Kahan bidiagonalization (GK-bidiagonalization) as well as Ritz vectors, we propose two methods for solving these problems in a fast and accurate way. Our experiments show the advantages of the proposed methods compared to the traditional and randomized singular value decomposition methods. The proposed methods are appropriate for applications involving huge matrices where the accuracy of the desired singular values and also all of their corresponding singular vectors are essential. As a real application, we evaluate the performance of our methods on the problem of Riemannian similarity learning between two various image datasets of MNIST and USPS.