The Fast Convergence of Incremental PCA
This work offers theoretical guarantees for incremental PCA, which is useful for applications like online learning and data streaming, but it is incremental as it builds on classical methods.
The paper tackles the problem of computing the top eigenvector of a covariance matrix from streaming data with incremental PCA algorithms, providing finite-sample convergence rates for Krasulina's and Oja's methods.
We consider a situation in which we see samples in $\mathbb{R}^d$ drawn i.i.d. from some distribution with mean zero and unknown covariance A. We wish to compute the top eigenvector of A in an incremental fashion - with an algorithm that maintains an estimate of the top eigenvector in O(d) space, and incrementally adjusts the estimate with each new data point that arrives. Two classical such schemes are due to Krasulina (1969) and Oja (1983). We give finite-sample convergence rates for both.