The Exponentially Tilted Gaussian Prior for Variational Autoencoders
This addresses safety concerns for deploying models in real-world applications by improving out-of-distribution detection, though it is an incremental improvement as a simple modification to existing VAE priors.
The paper tackles the problem of poor out-of-distribution detection in probabilistic generative models by proposing an exponentially tilted Gaussian prior for Variational Autoencoders, achieving state-of-the-art results on the AUC-ROC metric using log-likelihood.
An important property for deep neural networks is the ability to perform robust out-of-distribution detection on previously unseen data. This property is essential for safety purposes when deploying models for real world applications. Recent studies show that probabilistic generative models can perform poorly on this task, which is surprising given that they seek to estimate the likelihood of training data. To alleviate this issue, we propose the exponentially tilted Gaussian prior distribution for the Variational Autoencoder (VAE) which pulls points onto the surface of a hyper-sphere in latent space. This achieves state-of-the art results on the area under the curve-receiver operator characteristics metric using just the log-likelihood that the VAE naturally assigns. Because this prior is a simple modification of the traditional VAE prior, it is faster and easier to implement than competitive methods.