Learning Feature Sparse Principal Components
This addresses a computational bottleneck in high-dimensional data analysis for researchers and practitioners, though it is incremental as it builds on existing FSPCA methods.
The paper tackles the feature-sparsity constrained PCA (FSPCA) problem, which combines feature selection and PCA, by proposing two new algorithms: one solves it globally for low-rank covariances, and another approximates it for general covariances with theoretical guarantees, showing promising performance in experiments.
This paper presents new algorithms to solve the feature-sparsity constrained PCA problem (FSPCA), which performs feature selection and PCA simultaneously. Existing optimization methods for FSPCA require data distribution assumptions and are lack of global convergence guarantee. Though the general FSPCA problem is NP-hard, we show that, for a low-rank covariance, FSPCA can be solved globally (Algorithm 1). Then, we propose another strategy (Algorithm 2) to solve FSPCA for the general covariance by iteratively building a carefully designed proxy. We prove theoretical guarantees on approximation and convergence for the new algorithms. Experimental results show the promising performance of the new algorithms compared with the state-of-the-arts on both synthetic and real-world datasets.