The Expressive Power of a Class of Normalizing Flow Models
This addresses a gap in theoretical understanding for researchers in generative modeling, but it is incremental as it focuses on basic flows without proposing new methods.
The paper tackled the problem of formally understanding the representation power of basic normalizing flow models, finding that while they are highly expressive in one dimension, their power is limited in higher dimensions, especially with moderate depth.
Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.