Variantional autoencoder with decremental information bottleneck for disentanglement
This addresses the information diffusion problem in disentanglement learning for machine learning researchers, though it appears incremental as it builds on prior VAE methods.
The paper tackles the trade-off between disentanglement and reconstruction fidelity in variational autoencoders by introducing DeVAE, a framework using hierarchical latent spaces with decreasing information bottlenecks and disentanglement-invariant transformations, achieving a balance as demonstrated on dSprites and Shapes3D datasets.
One major challenge of disentanglement learning with variational autoencoders is the trade-off between disentanglement and reconstruction fidelity. Previous studies, which increase the information bottleneck during training, tend to lose the constraint of disentanglement, leading to the information diffusion problem. In this paper, we present a novel framework for disentangled representation learning, DeVAE, which utilizes hierarchical latent spaces with decreasing information bottlenecks across these spaces. The key innovation of our approach lies in connecting the hierarchical latent spaces through disentanglement-invariant transformations, allowing the sharing of disentanglement properties among spaces while maintaining an acceptable level of reconstruction performance. We demonstrate the effectiveness of DeVAE in achieving a balance between disentanglement and reconstruction through a series of experiments and ablation studies on dSprites and Shapes3D datasets. Code is available at https://github.com/erow/disentanglement_lib/tree/pytorch#devae.