Mixed-curvature Variational Autoencoders
This provides a generic approach for machine learning applications dealing with diverse data types, though it is incremental as it builds on existing curved-space VAEs.
The paper tackles the lack of a unified generative model for data with varying geometric biases by developing a Mixed-curvature Variational Autoencoder, which generalizes VAEs to product manifolds with constant curvatures, recovering Euclidean VAEs as a special case.
Euclidean geometry has historically been the typical "workhorse" for machine learning applications due to its power and simplicity. However, it has recently been shown that geometric spaces with constant non-zero curvature improve representations and performance on a variety of data types and downstream tasks. Consequently, generative models like Variational Autoencoders (VAEs) have been successfully generalized to elliptical and hyperbolic latent spaces. While these approaches work well on data with particular kinds of biases e.g. tree-like data for a hyperbolic VAE, there exists no generic approach unifying and leveraging all three models. We develop a Mixed-curvature Variational Autoencoder, an efficient way to train a VAE whose latent space is a product of constant curvature Riemannian manifolds, where the per-component curvature is fixed or learnable. This generalizes the Euclidean VAE to curved latent spaces and recovers it when curvatures of all latent space components go to 0.