Minimax Rates of Estimation for Sparse PCA in High Dimensions
This work addresses the challenge of sparse PCA estimation for statisticians and machine learning practitioners in high-dimensional data analysis, providing foundational theoretical guarantees.
The paper tackles the problem of estimating the leading eigenvector in sparse principal components analysis (PCA) under high-dimensional settings where variables exceed observations, proving optimal non-asymptotic minimax bounds for estimation error when the eigenvector lies in an ℓ_q ball for q in [0,1], with sharp bounds in p and n across a broad class of distributions.
We study sparse principal components analysis in the high-dimensional setting, where $p$ (the number of variables) can be much larger than $n$ (the number of observations). We prove optimal, non-asymptotic lower and upper bounds on the minimax estimation error for the leading eigenvector when it belongs to an $\ell_q$ ball for $q \in [0,1]$. Our bounds are sharp in $p$ and $n$ for all $q \in [0, 1]$ over a wide class of distributions. The upper bound is obtained by analyzing the performance of $\ell_q$-constrained PCA. In particular, our results provide convergence rates for $\ell_1$-constrained PCA.