GCVAE: Generalized-Controllable Variational AutoEncoder
This addresses a key bottleneck in unsupervised disentanglement learning for ML researchers, though it appears incremental as it builds on existing VAE variants.
The paper tackles the trade-off between reconstruction error and disentanglement in variational autoencoders by introducing a generalized framework with controllable hyperparameters, demonstrating state-of-the-art disentanglement performance while balancing reconstruction.
Variational autoencoders (VAEs) have recently been used for unsupervised disentanglement learning of complex density distributions. Numerous variants exist to encourage disentanglement in latent space while improving reconstruction. However, none have simultaneously managed the trade-off between attaining extremely low reconstruction error and a high disentanglement score. We present a generalized framework to handle this challenge under constrained optimization and demonstrate that it outperforms state-of-the-art existing models as regards disentanglement while balancing reconstruction. We introduce three controllable Lagrangian hyperparameters to control reconstruction loss, KL divergence loss and correlation measure. We prove that maximizing information in the reconstruction network is equivalent to information maximization during amortized inference under reasonable assumptions and constraint relaxation.