Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders
This work addresses the trade-offs in enforcing disentanglement for interpretability in autoencoders, but it is incremental as it quantifies effects without introducing new methods.
The paper investigates the impact of the disentanglement parameter β on variational autoencoders, finding that consistent variance exists in disentanglement measures and that disentanglement negatively affects the model's discriminative ability, especially with varying training data sizes.
The notion of disentangled autoencoders was proposed as an extension to the variational autoencoder by introducing a disentanglement parameter $β$, controlling the learning pressure put on the possible underlying latent representations. For certain values of $β$ this kind of autoencoders is capable of encoding independent input generative factors in separate elements of the code, leading to a more interpretable and predictable model behaviour. In this paper we quantify the effects of the parameter $β$ on the model performance and disentanglement. After training multiple models with the same value of $β$, we establish the existence of consistent variance in one of the disentanglement measures, proposed in literature. The negative consequences of the disentanglement to the autoencoder's discriminative ability are also asserted while varying the amount of examples available during training.