tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder
This work addresses the limitation of standard VAEs in handling structured data for unsupervised representation learning, though it appears incremental as it extends existing VAE frameworks.
The authors tackled the problem of VAEs ignoring data structure by proposing tvGP-VAE, which uses tensor-variate Gaussian processes to model correlations in latent variables, resulting in improved reconstruction performance for spatiotemporally correlated image time series.
Variational autoencoders (VAEs) are a powerful class of deep generative latent variable model for unsupervised representation learning on high-dimensional data. To ensure computational tractability, VAEs are often implemented with a univariate standard Gaussian prior and a mean-field Gaussian variational posterior distribution. This results in a vector-valued latent variables that are agnostic to the original data structure which might be highly correlated across and within multiple dimensions. We propose a tensor-variate extension to the VAE framework, the tensor-variate Gaussian process prior variational autoencoder (tvGP-VAE), which replaces the standard univariate Gaussian prior and posterior distributions with tensor-variate Gaussian processes. The tvGP-VAE is able to explicitly model correlation structures via the use of kernel functions over the dimensions of tensor-valued latent variables. Using spatiotemporally correlated image time series as an example, we show that the choice of which correlation structures to explicitly represent in the latent space has a significant impact on model performance in terms of reconstruction.