Stochastic Backpropagation and Approximate Inference in Deep Generative Models
This work addresses the problem of efficient Bayesian inference in complex generative models for researchers and practitioners in machine learning, representing a novel integration of deep learning and approximate inference rather than an incremental improvement.
The paper tackles the challenge of scalable inference and learning in deep generative models by introducing a recognition model for approximate posterior distributions and developing stochastic backpropagation for joint optimization. The result is a model that generates realistic samples, accurately imputes missing data, and aids in high-dimensional data visualization on real-world datasets.
We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.