Geodesic Clustering in Deep Generative Models
This addresses the challenge of performing accurate clustering in deep generative models for researchers and practitioners in unsupervised learning, though it is incremental as it builds on existing geometric insights.
The paper tackles the problem of distorted pairwise distances in latent representations of deep generative models, which hinders unsupervised clustering, by proposing an efficient algorithm for computing geodesic distances that account for the model's geometry, resulting in geodesic distances that reflect the internal data structure.
Deep generative models are tremendously successful in learning low-dimensional latent representations that well-describe the data. These representations, however, tend to much distort relationships between points, i.e. pairwise distances tend to not reflect semantic similarities well. This renders unsupervised tasks, such as clustering, difficult when working with the latent representations. We demonstrate that taking the geometry of the generative model into account is sufficient to make simple clustering algorithms work well over latent representations. Leaning on the recent finding that deep generative models constitute stochastically immersed Riemannian manifolds, we propose an efficient algorithm for computing geodesics (shortest paths) and computing distances in the latent space, while taking its distortion into account. We further propose a new architecture for modeling uncertainty in variational autoencoders, which is essential for understanding the geometry of deep generative models. Experiments show that the geodesic distance is very likely to reflect the internal structure of the data.