Universal Approximation of Residual Flows in Maximum Mean Discrepancy
This work addresses a theoretical gap for researchers in deep generative models, though it appears incremental as it builds on existing flow architectures.
The authors tackled the lack of theoretical understanding of the expressiveness of normalizing flows by proving that residual flows, a class of normalizing flows with Lipschitz residual blocks, are universal approximators in maximum mean discrepancy, and they provided upper bounds on the number of blocks needed for approximation.
Normalizing flows are a class of flexible deep generative models that offer easy likelihood computation. Despite their empirical success, there is little theoretical understanding of their expressiveness. In this work, we study residual flows, a class of normalizing flows composed of Lipschitz residual blocks. We prove residual flows are universal approximators in maximum mean discrepancy. We provide upper bounds on the number of residual blocks to achieve approximation under different assumptions.