Tutorial: Complexity analysis of Singular Value Decomposition and its variants
This is an incremental tutorial for researchers and practitioners in numerical linear algebra and machine learning.
The paper compared the time and space complexity of Singular Value Decomposition (SVD) and its variants, including truncated SVD, Krylov method, and Randomized PCA, for calculating all eigenpairs, and discussed the relationship between SVD and Principal Component Analysis.
We compared the regular Singular Value Decomposition (SVD), truncated SVD, Krylov method and Randomized PCA, in terms of time and space complexity. It is well-known that Krylov method and Randomized PCA only performs well when k << n, i.e. the number of eigenpair needed is far less than that of matrix size. We compared them for calculating all the eigenpairs. We also discussed the relationship between Principal Component Analysis and SVD.