Classification of high-dimensional data with spiked covariance matrix structure
This addresses classification challenges in high-dimensional statistics for researchers and practitioners, though it appears incremental as it builds on existing whitening and screening techniques.
The paper tackles high-dimensional classification with spiked covariance matrices and sparse whitened mean differences by proposing an adaptive classifier that whitens data, screens features, and applies Fisher discriminant analysis. The method achieves Bayes optimality under certain asymptotic conditions and shows competitive performance with state-of-the-art methods on real and synthetic data while using lower-dimensional representations.
We study the classification problem for high-dimensional data with $n$ observations on $p$ features where the $p \times p$ covariance matrix $Σ$ exhibits a spiked eigenvalue structure and the vector $ζ$, given by the difference between the {\em whitened} mean vectors, is sparse. We analyze an adaptive classifier (adaptive with respect to the sparsity $s$) that first performs dimension reduction on the feature vectors prior to classification in the dimensionally reduced space, i.e., the classifier whitens the data, then screens the features by keeping only those corresponding to the $s$ largest coordinates of $ζ$ and finally applies Fisher linear discriminant on the selected features. Leveraging recent results on entrywise matrix perturbation bounds for covariance matrices, we show that the resulting classifier is Bayes optimal whenever $n \rightarrow \infty$ and $s \sqrt{n^{-1} \ln p} \rightarrow 0$. Notably, our theory also guarantees Bayes optimality for the corresponding quadratic discriminant analysis (QDA). Experimental results on real and synthetic data further indicate that the proposed approach is competitive with state-of-the-art methods while operating on a substantially lower-dimensional representation.