Copula-like Variational Inference
This provides an incremental improvement for practitioners in Bayesian machine learning needing more flexible variational approximations.
The paper tackles the problem of approximating complex posterior distributions in variational inference by introducing a new family of variational distributions based on copula-like densities with efficient sampling. The method performs comparably to state-of-the-art variational approximations on standard benchmarks for Bayesian Neural Networks.
This paper considers a new family of variational distributions motivated by Sklar's theorem. This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i.e. with a complexity linear in the dimension of state space. Then, the proposed variational densities that we suggest can be seen as arising from these copula-like densities used as base distributions on the hypercube with Gaussian quantile functions and sparse rotation matrices as normalizing flows. The latter correspond to a rotation of the marginals with complexity $\mathcal{O}(d \log d)$. We provide some empirical evidence that such a variational family can also approximate non-Gaussian posteriors and can be beneficial compared to Gaussian approximations. Our method performs largely comparably to state-of-the-art variational approximations on standard regression and classification benchmarks for Bayesian Neural Networks.