IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
This work addresses the problem of learning interpretable latent representations for generative modeling, offering an incremental improvement over existing methods like InfoGAN and β-VAEs.
The paper tackles unsupervised disentangled representation learning by proposing IB-GAN, which integrates the Information Bottleneck framework into GAN optimization to constrain mutual information between input and output, resulting in competitive disentanglement scores on dSprites and Color-dSprites datasets and better FID scores on CelebA and 3D Chairs compared to β-VAEs and InfoGAN.
We propose a new GAN-based unsupervised model for disentangled representation learning. The new model is discovered in an attempt to utilize the Information Bottleneck (IB) framework to the optimization of GAN, thereby named IB-GAN. The architecture of IB-GAN is partially similar to that of InfoGAN but has a critical difference; an intermediate layer of the generator is leveraged to constrain the mutual information between the input and the generated output. The intermediate stochastic layer can serve as a learnable latent distribution that is trained with the generator jointly in an end-to-end fashion. As a result, the generator of IB-GAN can harness the latent space in a disentangled and interpretable manner. With the experiments on dSprites and Color-dSprites dataset, we demonstrate that IB-GAN achieves competitive disentanglement scores to those of state-of-the-art \b{eta}-VAEs and outperforms InfoGAN. Moreover, the visual quality and the diversity of samples generated by IB-GAN are often better than those by \b{eta}-VAEs and Info-GAN in terms of FID score on CelebA and 3D Chairs dataset.