SVD update methods for large matrices and applications
For practitioners needing to incrementally update SVD decompositions in large-scale PCA or nullspace computations, this method offers computational savings over recomputing from scratch.
This paper presents an efficient method for updating the SVD of a tall-thin matrix when augmenting it with additional columns, enabling fast computation of dominant singular vectors for PCA and kernel basis extraction. The method is demonstrated on two concrete applications.
We consider the problem of updating the SVD when augmenting a "tall thin" matrix, i.e., a rectangular matrix $A \in \RR^{m \times n}$ with $m \gg n$. Supposing that an SVD of $A$ is already known, and given a matrix $B \in \RR^{m \times n'}$, we derive an efficient method to compute and efficiently store the SVD of the augmented matrix $[ A B ] \in \RR^{m \times (n+n')}$. This is an important tool for two types of applications: in the context of principal component analysis, the dominant left singular vectors provided by this decomposition form an orthonormal basis for the best linear subspace of a given dimension, while from the right singular vectors one can extract an orthonormal basis of the kernel of the matrix. We also describe two concrete applications of these concepts which motivated the development of our method and to which it is very well adapted.