DAVA: Disentangling Adversarial Variational Autoencoder
This addresses the challenge of dataset-specific hyperparameter selection in unsupervised disentanglement, which is incremental as it builds on existing variational autoencoder methods.
The paper tackles the problem of hyperparameter sensitivity in disentangled representation learning for variational autoencoders by introducing DAVA, a training procedure that eliminates the need for hyperparameter tuning and achieves competitive performance across diverse datasets without any tuning.
The use of well-disentangled representations offers many advantages for downstream tasks, e.g. an increased sample efficiency, or better interpretability. However, the quality of disentangled interpretations is often highly dependent on the choice of dataset-specific hyperparameters, in particular the regularization strength. To address this issue, we introduce DAVA, a novel training procedure for variational auto-encoders. DAVA completely alleviates the problem of hyperparameter selection. We compare DAVA to models with optimal hyperparameters. Without any hyperparameter tuning, DAVA is competitive on a diverse range of commonly used datasets. Underlying DAVA, we discover a necessary condition for unsupervised disentanglement, which we call PIPE. We demonstrate the ability of PIPE to positively predict the performance of downstream models in abstract reasoning. We also thoroughly investigate correlations with existing supervised and unsupervised metrics. The code is available at https://github.com/besterma/dava.