Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows
This work addresses theoretical limitations in manifold learning and density estimation for machine learning, providing foundational insights into the capabilities of injective flows.
The paper tackles the problem of approximating probability measures on manifolds using injective flows, showing that these flows can universally approximate measures on certain embeddings when the embedding dimension is sufficiently large (m ≥ 3n+1), resolving a prior conjecture.
We study approximation of probability measures supported on $n$-dimensional manifolds embedded in $\mathbb{R}^m$ by injective flows -- neural networks composed of invertible flows and injective layers. We show that in general, injective flows between $\mathbb{R}^n$ and $\mathbb{R}^m$ universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, $m \ge 3n+1$, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer and Cranmer 2020.