Sylvester Normalizing Flows for Variational Inference
This work addresses a bottleneck in variational inference for machine learning practitioners, offering an incremental improvement over existing normalizing flow methods.
The paper tackles the problem of limited flexibility in variational inference by introducing Sylvester normalizing flows, which generalize planar flows to remove a single-unit bottleneck, resulting in improved performance on several datasets compared to planar and inverse autoregressive flows.
Variational inference relies on flexible approximate posterior distributions. Normalizing flows provide a general recipe to construct flexible variational posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single-unit bottleneck from planar flows, making a single transformation much more flexible. We compare the performance of Sylvester normalizing flows against planar flows and inverse autoregressive flows and demonstrate that they compare favorably on several datasets.