A Closer Look at Disentangling in $β$-VAE
This addresses a theoretical limitation in disentangled representation learning for machine learning researchers, but it is incremental as it builds on existing β-VAE analysis.
The paper investigates the β-VAE model for learning disentangled representations, showing that its conditional independence constraint conflicts with statistical independence of latents, leading to non-monotonic inference performance with a finite optimal β.
In many data analysis tasks, it is beneficial to learn representations where each dimension is statistically independent and thus disentangled from the others. If data generating factors are also statistically independent, disentangled representations can be formed by Bayesian inference of latent variables. We examine a generalization of the Variational Autoencoder (VAE), $β$-VAE, for learning such representations using variational inference. $β$-VAE enforces conditional independence of its bottleneck neurons controlled by its hyperparameter $β$. This condition is in general not compatible with the statistical independence of latents. By providing analytical and numerical arguments, we show that this incompatibility leads to a non-monotonic inference performance in $β$-VAE with a finite optimal $β$.