Variational Auto-encoded Deep Gaussian Processes
This work addresses scalability issues in deep non-parametric generative models for researchers and practitioners in machine learning, representing an incremental improvement by combining existing concepts like variational inference and deep Gaussian processes.
The authors tackled the challenge of scaling deep Gaussian processes for large datasets by introducing a variational auto-encoded framework with a reparametrized recognition model, which prevents variational parameter proliferation and enables handling mainstream deep learning-sized datasets, demonstrating efficacy in deep unsupervised learning and Bayesian optimization.
We develop a scalable deep non-parametric generative model by augmenting deep Gaussian processes with a recognition model. Inference is performed in a novel scalable variational framework where the variational posterior distributions are reparametrized through a multilayer perceptron. The key aspect of this reformulation is that it prevents the proliferation of variational parameters which otherwise grow linearly in proportion to the sample size. We derive a new formulation of the variational lower bound that allows us to distribute most of the computation in a way that enables to handle datasets of the size of mainstream deep learning tasks. We show the efficacy of the method on a variety of challenges including deep unsupervised learning and deep Bayesian optimization.