Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?
This work provides practical guidance for researchers and practitioners in fields like industrial monitoring and computer vision who need efficient online PCA without storing all data, but it is incremental as it compares existing methods rather than introducing new ones.
The paper tackles the problem of selecting an appropriate online PCA algorithm for high-dimensional data by comparing perturbation techniques, incremental methods, and stochastic optimization based on statistical accuracy, computation time, and memory requirements, using both artificial and real data.
In the current context of data explosion, online techniques that do not require storing all data in memory are indispensable to routinely perform tasks like principal component analysis (PCA). Recursive algorithms that update the PCA with each new observation have been studied in various fields of research and found wide applications in industrial monitoring, computer vision, astronomy, and latent semantic indexing, among others. This work provides guidance for selecting an online PCA algorithm in practice. We present the main approaches to online PCA, namely, perturbation techniques, incremental methods, and stochastic optimization, and compare their statistical accuracy, computation time, and memory requirements using artificial and real data. Extensions to missing data and to functional data are discussed. All studied algorithms are available in the R package onlinePCA on CRAN.