Deep Diffusion Maps
This work addresses inefficiencies in manifold learning for researchers and practitioners dealing with large datasets, though it is incremental as it builds on existing Diffusion Maps with deep learning enhancements.
The authors tackled the computational and scalability limitations of Diffusion Maps for dimensionality reduction by reformulating it as an unconstrained minimization problem and training a neural network to compute embeddings, achieving comparable performance to Diffusion Maps and the Nyström method on synthetic and real datasets.
One of the fundamental problems within the field of machine learning is dimensionality reduction. Dimensionality reduction methods make it possible to combat the so-called curse of dimensionality, visualize high-dimensional data and, in general, improve the efficiency of storing and processing large data sets. One of the best-known nonlinear dimensionality reduction methods is Diffusion Maps. However, despite their virtues, both Diffusion Maps and many other manifold learning methods based on the spectral decomposition of kernel matrices have drawbacks such as the inability to apply them to data outside the initial set, their computational complexity, and high memory costs for large data sets. In this work, we propose to alleviate these problems by resorting to deep learning. Specifically, a new formulation of Diffusion Maps embedding is offered as a solution to a certain unconstrained minimization problem and, based on it, a cost function to train a neural network which computes Diffusion Maps embedding -- both inside and outside the training sample -- without the need to perform any spectral decomposition. The capabilities of this approach are compared on different data sets, both real and synthetic, with those of Diffusion Maps and the Nystrom method.