Sum-of-Squares Polynomial Flow
This work addresses density estimation for machine learning applications, offering a novel method that unifies existing approaches but is incremental in its improvements.
The paper tackles high-dimensional density estimation by proposing a unified framework based on triangular maps, which reveals connections between autoregressive and flow-based methods and introduces a new Sum-of-Squares (SOS) flow that is interpretable, universal, and easy to train. It achieves competitive results in simulations and real-world datasets.
Triangular map is a recent construct in probability theory that allows one to transform any source probability density function to any target density function. Based on triangular maps, we propose a general framework for high-dimensional density estimation, by specifying one-dimensional transformations (equivalently conditional densities) and appropriate conditioner networks. This framework (a) reveals the commonalities and differences of existing autoregressive and flow based methods, (b) allows a unified understanding of the limitations and representation power of these recent approaches and, (c) motivates us to uncover a new Sum-of-Squares (SOS) flow that is interpretable, universal, and easy to train. We perform several synthetic experiments on various density geometries to demonstrate the benefits (and short-comings) of such transformations. SOS flows achieve competitive results in simulations and several real-world datasets.