Manifold-Matching Autoencoders
This work addresses the challenge of preserving geometric structure in autoencoder representations, which is incremental as it builds on existing regularization techniques.
The paper tackles the problem of unsupervised regularization for autoencoders by introducing Manifold-Matching Autoencoders (MMAE), which align pairwise distances between latent and input spaces, resulting in improved performance on nearest-neighbor and persistent homology metrics compared to similar methods.
We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).