On the Optimality of the Oja's Algorithm for Online PCA
This provides a theoretical guarantee for efficient online PCA, which is incremental but addresses a bottleneck in streaming data analysis.
The paper proves that Oja's algorithm for online PCA achieves a gap-free, global convergence rate to approximate the principal component subspace for sub-Gaussian distributions, with its upper bound matching the lower bound of offline PCA up to a constant factor.
In this paper we analyze the behavior of the Oja's algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.