Graph Regularized PCA
This work addresses the need for structure-aware dimensionality reduction in high-dimensional data with feature dependencies, offering a practical and scalable method that improves structural fidelity without sacrificing predictive performance, though it is incremental as it builds on PCA with graph-based regularization.
The authors tackled the problem of PCA's suboptimality under non-isotropic noise by introducing Graph Regularized PCA (GR-PCA), which incorporates data dependency structures via graph regularization to yield interpretable principal components aligned with conditional relationships, showing advantages in variance concentration and structural fidelity over PCA in scenarios with graph-correlated high-frequency signals.
High-dimensional data often exhibit dependencies among variables that violate the isotropic-noise assumption under which principal component analysis (PCA) is optimal. For cases where the noise is not independent and identically distributed across features (i.e., the covariance is not spherical) we introduce Graph Regularized PCA (GR-PCA). It is a graph-based regularization of PCA that incorporates the dependency structure of the data features by learning a sparse precision graph and biasing loadings toward the low-frequency Fourier modes of the corresponding graph Laplacian. Consequently, high-frequency signals are suppressed, while graph-coherent low-frequency ones are preserved, yielding interpretable principal components aligned with conditional relationships. We evaluate GR-PCA on synthetic data spanning diverse graph topologies, signal-to-noise ratios, and sparsity levels. Compared to mainstream alternatives, it concentrates variance on the intended support, produces loadings with lower graph-Laplacian energy, and remains competitive in out-of-sample reconstruction. When high-frequency signals are present, the graph Laplacian penalty prevents overfitting, reducing the reconstruction accuracy but improving structural fidelity. The advantage over PCA is most pronounced when high-frequency signals are graph-correlated, whereas PCA remains competitive when such signals are nearly rotationally invariant. The procedure is simple to implement, modular with respect to the precision estimator, and scalable, providing a practical route to structure-aware dimensionality reduction that improves structural fidelity without sacrificing predictive performance.