Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspective
Provides theoretical guarantees for bi-Lipschitz normalizing flows, addressing a gap in understanding their approximation power for practitioners in generative modeling.
The paper characterizes the expressivity of bi-Lipschitz normalizing flows by linking them to score-based diffusion models via the probability flow ODE. It proves that bi-Lipschitz flows can universally approximate any probability density in L1, and for Gaussian convolution targets, achieve convergence in KL divergence without early stopping.
Many normalizing flow architectures impose regularity constraints, yet their distributional approximation properties are not fully characterized. We study the expressivity of bi-Lipschitz normalizing flows through the lens of score-based diffusion models. For the probability flow ODE of a variance-preserving diffusion, Lipschitz regularity of the score induces a flow of bi-Lipschitz diffeomorphic transport maps. This ODE bridge allows us to analyze the distributional approximation power of bi-Lipschitz normalizing flows and, conversely, derive deterministic convergence guarantees for diffusion-based transport. Our key idea is to use the probability flow ODE to link regularity of the score to regularity of the induced transport maps. We verify score regularity for broad target densities, including compactly supported densities, Gaussian convolutions of compactly supported measures and finite Gaussian mixtures. We obtain a universal distributional approximation result: Gaussian pullbacks induced by bi-Lipschitz variance-preserving transport maps are $L^1$-dense among all probability densities. For Gaussian convolution targets, we further obtain convergence in Kullback-Leibler divergence without early stopping.