Linearized Diffusion Map
This is an incremental improvement for researchers and practitioners in machine learning, offering a new linear dimensionality reduction technique with potential for interpretable analysis.
The authors tackled the problem of dimensionality reduction by introducing Linearized Diffusion Map (LDM), a linear method that approximates diffusion-map kernels, and found that LDM outperforms PCA in capturing manifold structures in high-dimensional datasets like MNIST and COIL-20, while PCA is better for variance-dominated scenarios.
We introduce the Linearized Diffusion Map (LDM), a novel linear dimensionality reduction method constructed via a linear approximation of the diffusion-map kernel. LDM integrates the geometric intuition of diffusion-based nonlinear methods with the computational simplicity, efficiency, and interpretability inherent in linear embeddings such as PCA and classical MDS. Through comprehensive experiments on synthetic datasets (Swiss roll and hyperspheres) and real-world benchmarks (MNIST and COIL-20), we illustrate that LDM captures distinct geometric features of datasets compared to PCA, offering complementary advantages. Specifically, LDM embeddings outperform PCA in datasets exhibiting explicit manifold structures, particularly in high-dimensional regimes, whereas PCA remains preferable in scenarios dominated by variance or noise. Furthermore, the complete positivity of LDM's kernel matrix allows direct applicability of Non-negative Matrix Factorization (NMF), suggesting opportunities for interpretable latent-structure discovery. Our analysis positions LDM as a valuable new linear dimensionality reduction technique with promising theoretical and practical extensions.