Increasing Expressivity of a Hyperspherical VAE
This work addresses a bottleneck in latent representation learning for machine learning applications, but it is incremental as it builds on prior hyperspherical methods.
The authors tackled the limited expressivity of hyperspherical VAEs in high dimensions by proposing a product-space parameterization, showing improved results on image datasets.
Learning suitable latent representations for observed, high-dimensional data is an important research topic underlying many recent advances in machine learning. While traditionally the Gaussian normal distribution has been the go-to latent parameterization, recently a variety of works have successfully proposed the use of manifold-valued latents. In one such work (Davidson et al., 2018), the authors empirically show the potential benefits of using a hyperspherical von Mises-Fisher (vMF) distribution in low dimensionality. However, due to the unique distributional form of the vMF, expressivity in higher dimensional space is limited as a result of its scalar concentration parameter leading to a 'hyperspherical bottleneck'. In this work we propose to extend the usability of hyperspherical parameterizations to higher dimensions using a product-space instead, showing improved results on a selection of image datasets.