Hyperspherical Variational Auto-Encoders
This addresses a limitation in unsupervised learning for data with hyperspherical latent structures, though it is an incremental improvement over existing VAE methods.
The paper tackles the problem that Gaussian distributions in Variational Auto-Encoders (VAEs) fail to model data with a latent hyperspherical structure, and proposes using a von Mises-Fisher distribution to create a hyperspherical VAE (S-VAE), which outperforms normal VAEs in low dimensions on various data types.
The Variational Auto-Encoder (VAE) is one of the most used unsupervised machine learning models. But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails to model data with a latent hyperspherical structure. To address this issue we propose using a von Mises-Fisher (vMF) distribution instead, leading to a hyperspherical latent space. Through a series of experiments we show how such a hyperspherical VAE, or $\mathcal{S}$-VAE, is more suitable for capturing data with a hyperspherical latent structure, while outperforming a normal, $\mathcal{N}$-VAE, in low dimensions on other data types. Code at http://github.com/nicola-decao/s-vae-tf and https://github.com/nicola-decao/s-vae-pytorch