Disentanglement with Hyperspherical Latent Spaces using Diffusion Variational Autoencoders
This work addresses a specific challenge in representation learning for data with complex geometries, offering an incremental improvement over existing methods.
The paper tackles the problem of learning disentangled representations when underlying factors have non-Euclidean geometries, such as periodic structures, by proposing a Diffusion Variational Autoencoder with a hyperspherical latent space, which recovers these factors effectively.
A disentangled representation of a data set should be capable of recovering the underlying factors that generated it. One question that arises is whether using Euclidean space for latent variable models can produce a disentangled representation when the underlying generating factors have a certain geometrical structure. Take for example the images of a car seen from different angles. The angle has a periodic structure but a 1-dimensional representation would fail to capture this topology. How can we address this problem? The submissions presented for the first stage of the NeurIPS2019 Disentanglement Challenge consist of a Diffusion Variational Autoencoder ($Δ$VAE) with a hyperspherical latent space which can, for example, recover periodic true factors. The training of the $Δ$VAE is enhanced by incorporating a modified version of the Evidence Lower Bound (ELBO) for tailoring the encoding capacity of the posterior approximate.