Spike and slab Bayesian sparse principal component analysis
This work addresses the problem of dimensionality reduction in high-dimensional data for researchers and practitioners in statistics and machine learning, offering a scalable Bayesian SPCA method with theoretical guarantees, though it is incremental as it builds on prior SPCA and variational inference techniques.
The authors tackled the challenge of developing a theoretically justified and computationally scalable Bayesian sparse principal component analysis (SPCA) method by proposing a novel parameter-expanded coordinate ascent variational inference (PX-CAVI) algorithm with a spike and slab prior, which outperformed existing SPCA approaches in numerical simulations and was applied to a lung cancer gene expression dataset.
Sparse principal component analysis (SPCA) is a popular tool for dimensionality reduction in high-dimensional data. However, there is still a lack of theoretically justified Bayesian SPCA methods that can scale well computationally. One of the major challenges in Bayesian SPCA is selecting an appropriate prior for the loadings matrix, considering that principal components are mutually orthogonal. We propose a novel parameter-expanded coordinate ascent variational inference (PX-CAVI) algorithm. This algorithm utilizes a spike and slab prior, which incorporates parameter expansion to cope with the orthogonality constraint. Besides comparing to two popular SPCA approaches, we introduce the PX-EM algorithm as an EM analogue to the PX-CAVI algorithm for comparison. Through extensive numerical simulations, we demonstrate that the PX-CAVI algorithm outperforms these SPCA approaches, showcasing its superiority in terms of performance. We study the posterior contraction rate of the variational posterior, providing a novel contribution to the existing literature. The PX-CAVI algorithm is then applied to study a lung cancer gene expression dataset. The R package VBsparsePCA with an implementation of the algorithm is available on the Comprehensive R Archive Network (CRAN).