Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes
This addresses the limitation of autoencoders in modeling sample correlations for representation learning and generative modeling, though it appears incremental as it builds on existing Bayesian and Gaussian Process methods.
The authors tackled the problem of autoencoders failing to capture correlations between data samples by proposing a fully Bayesian autoencoder with sparse Gaussian Process priors on the latent space, showing it consistently outperforms Variational Autoencoder alternatives on representation learning and generative modeling tasks.
Autoencoders and their variants are among the most widely used models in representation learning and generative modeling. However, autoencoder-based models usually assume that the learned representations are i.i.d. and fail to capture the correlations between the data samples. To address this issue, we propose a novel Sparse Gaussian Process Bayesian Autoencoder (SGPBAE) model in which we impose fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder. We perform posterior estimation for this model via stochastic gradient Hamiltonian Monte Carlo. We evaluate our approach qualitatively and quantitatively on a wide range of representation learning and generative modeling tasks and show that our approach consistently outperforms multiple alternatives relying on Variational Autoencoders.