Discretely Indexed Flows
This work addresses a bottleneck in variational estimation for machine learning practitioners by enhancing the ability to model intricate distributions, though it is incremental as it builds on existing Normalizing Flows and mixture models.
The authors tackled the problem of capturing complex distributions with discontinuities and sharp edges by proposing Discretely Indexed Flows (DIF), which extend Normalizing Flows with stochastic, discretely indexed transports, resulting in a model that maintains tractable density and sampling while improving flexibility for variational inference and density estimation.
In this paper we propose Discretely Indexed flows (DIF) as a new tool for solving variational estimation problems. Roughly speaking, DIF are built as an extension of Normalizing Flows (NF), in which the deterministic transport becomes stochastic, and more precisely discretely indexed. Due to the discrete nature of the underlying additional latent variable, DIF inherit the good computational behavior of NF: they benefit from both a tractable density as well as a straightforward sampling scheme, and can thus be used for the dual problems of Variational Inference (VI) and of Variational density estimation (VDE). On the other hand, DIF can also be understood as an extension of mixture density models, in which the constant mixture weights are replaced by flexible functions. As a consequence, DIF are better suited for capturing distributions with discontinuities, sharp edges and fine details, which is a main advantage of this construction. Finally we propose a methodology for constructiong DIF in practice, and see that DIF can be sequentially cascaded, and cascaded with NF.