Hyperspherical Variational Autoencoders Using Efficient Spherical Cauchy Distribution
This work addresses a specific bottleneck in generative modeling for directional data, offering an incremental improvement over existing methods like von Mises-Fisher distributions.
The authors tackled the problem of variational autoencoders (VAEs) with hyperspherical latent spaces by proposing a spherical Cauchy distribution, which better captures directional data and avoids numerical instabilities, resulting in a more efficient and stable training process.
We propose a novel variational autoencoder (VAE) architecture that employs a spherical Cauchy (spCauchy) latent distribution. Unlike traditional Gaussian latent spaces or the widely used von Mises-Fisher (vMF) distribution, spCauchy provides a more natural hyperspherical representation of latent variables, better capturing directional data while maintaining flexibility. Its heavy-tailed nature prevents over-regularization, ensuring efficient latent space utilization while offering a more expressive representation. Additionally, spCauchy circumvents the numerical instabilities inherent to vMF, which arise from computing normalization constants involving Bessel functions. Instead, it enables a fully differentiable and efficient reparameterization trick via Möbius transformations, allowing for stable and scalable training. The KL divergence can be computed through a rapidly converging power series, eliminating concerns of underflow or overflow associated with evaluation of ratios of hypergeometric functions. These properties make spCauchy a compelling alternative for VAEs, offering both theoretical advantages and practical efficiency in high-dimensional generative modeling.