Convolutional Normalizing Flows
This work addresses a computational bottleneck in variational inference for machine learning practitioners, though it appears incremental as it builds on existing normalizing flow methods.
The paper tackles the trade-off between flexibility and computational cost in normalizing flows for Bayesian posterior inference by proposing ConvFlow, a convolutional architecture, which demonstrates effectiveness and efficiency in experiments on synthetic and real-world problems.
Bayesian posterior inference is prevalent in various machine learning problems. Variational inference provides one way to approximate the posterior distribution, however its expressive power is limited and so is the accuracy of resulting approximation. Recently, there has a trend of using neural networks to approximate the variational posterior distribution due to the flexibility of neural network architecture. One way to construct flexible variational distribution is to warp a simple density into a complex by normalizing flows, where the resulting density can be analytically evaluated. However, there is a trade-off between the flexibility of normalizing flow and computation cost for efficient transformation. In this paper, we propose a simple yet effective architecture of normalizing flows, ConvFlow, based on convolution over the dimensions of random input vector. Experiments on synthetic and real world posterior inference problems demonstrate the effectiveness and efficiency of the proposed method.