A Geometric Perspective on Autoencoders
This work addresses a fundamental problem in unsupervised learning for researchers and practitioners using autoencoders, though it appears incremental as it builds on existing geometric perspectives.
The paper tackles the problem of autoencoders learning incorrect manifolds and distorted latent spaces due to multiple possible solutions, and introduces geometric approaches to address these issues.
This paper presents the geometric aspect of the autoencoder framework, which, despite its importance, has been relatively less recognized. Given a set of high-dimensional data points that approximately lie on some lower-dimensional manifold, an autoencoder learns the \textit{manifold} and its \textit{coordinate chart}, simultaneously. This geometric perspective naturally raises inquiries like "Does a finite set of data points correspond to a single manifold?" or "Is there only one coordinate chart that can represent the manifold?". The responses to these questions are negative, implying that there are multiple solution autoencoders given a dataset. Consequently, they sometimes produce incorrect manifolds with severely distorted latent space representations. In this paper, we introduce recent geometric approaches that address these issues.