Flows for simultaneous manifold learning and density estimation
This work addresses the challenge of representing datasets with manifold structure more faithfully, which is important for researchers and practitioners in machine learning, particularly for tasks like dimensionality reduction, denoising, and out-of-distribution detection, and it is incremental as it builds on existing generative models.
The authors tackled the problem of simultaneously learning data manifolds and probability densities on them by introducing manifold-learning flows (M-flows), a new class of generative models that combine aspects of normalizing flows, GANs, autoencoders, and energy-based models, resulting in better inference than standard flows in ambient data space as demonstrated in experiments.
We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality reduction, denoising, and out-of-distribution detection. We argue why such models should not be trained by maximum likelihood alone and present a new training algorithm that separates manifold and density updates. In a range of experiments we demonstrate how M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.