Amortized Population Gibbs Samplers with Neural Sufficient Statistics
This addresses the challenge of training highly structured deep generative models in an unsupervised manner, representing an incremental advancement in variational inference techniques.
The paper tackles the problem of scalable structured variational inference by introducing amortized population Gibbs samplers, which frame it as adaptive importance sampling and use neural sufficient statistics, resulting in substantial improvements in inference accuracy over standard autoencoding variational methods.
We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frames structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can train highly structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.