Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows
This addresses a limitation in generative modeling for domains like image data where data lies on manifolds, offering a tractable solution that is incremental by building on existing flow methods.
The paper tackles the problem of modeling data on unknown low-dimensional manifolds with normalizing flows, which typically fail to provide tractable densities in such cases, and introduces Conformal Embedding Flows to enable exact density estimation on learned manifolds, demonstrating effectiveness on synthetic and real-world data.
Normalizing flows are generative models that provide tractable density estimation via an invertible transformation from a simple base distribution to a complex target distribution. However, this technique cannot directly model data supported on an unknown low-dimensional manifold, a common occurrence in real-world domains such as image data. Recent attempts to remedy this limitation have introduced geometric complications that defeat a central benefit of normalizing flows: exact density estimation. We recover this benefit with Conformal Embedding Flows, a framework for designing flows that learn manifolds with tractable densities. We argue that composing a standard flow with a trainable conformal embedding is the most natural way to model manifold-supported data. To this end, we present a series of conformal building blocks and apply them in experiments with synthetic and real-world data to demonstrate that flows can model manifold-supported distributions without sacrificing tractable likelihoods.