Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference
This work addresses scalability issues in unsupervised dimensionality reduction for high-dimensional data, though it appears incremental as it builds on existing GPLVM methods.
The authors tackled the scalability limitations of Bayesian Gaussian Process Latent Variable Models (GPLVM) by introducing a doubly stochastic variational inference framework that enables minibatch training, achieving high-fidelity reconstructions even with massively missing data.
Gaussian process latent variable models (GPLVM) are a flexible and non-linear approach to dimensionality reduction, extending classical Gaussian processes to an unsupervised learning context. The Bayesian incarnation of the GPLVM Titsias and Lawrence, 2010] uses a variational framework, where the posterior over latent variables is approximated by a well-behaved variational family, a factorized Gaussian yielding a tractable lower bound. However, the non-factories ability of the lower bound prevents truly scalable inference. In this work, we study the doubly stochastic formulation of the Bayesian GPLVM model amenable with minibatch training. We show how this framework is compatible with different latent variable formulations and perform experiments to compare a suite of models. Further, we demonstrate how we can train in the presence of massively missing data and obtain high-fidelity reconstructions. We demonstrate the model's performance by benchmarking against the canonical sparse GPLVM for high-dimensional data examples.