HSIC-InfoGAN: Learning Unsupervised Disentangled Representations by Maximising Approximated Mutual Information
This work addresses the challenge of unsupervised disentanglement for researchers in machine learning, but it is incremental as it builds on the existing InfoGAN framework with a different approximation method.
The paper tackles the problem of learning unsupervised disentangled representations by proposing HSIC-InfoGAN, which uses the Hilbert-Schmidt Independence Criterion to approximate mutual information, eliminating the need for an auxiliary network. The result is a qualitative comparison of disentanglement levels and a hyperparameter tuning strategy, with potential applications in medical domains.
Learning disentangled representations requires either supervision or the introduction of specific model designs and learning constraints as biases. InfoGAN is a popular disentanglement framework that learns unsupervised disentangled representations by maximising the mutual information between latent representations and their corresponding generated images. Maximisation of mutual information is achieved by introducing an auxiliary network and training with a latent regression loss. In this short exploratory paper, we study the use of the Hilbert-Schmidt Independence Criterion (HSIC) to approximate mutual information between latent representation and image, termed HSIC-InfoGAN. Directly optimising the HSIC loss avoids the need for an additional auxiliary network. We qualitatively compare the level of disentanglement in each model, suggest a strategy to tune the hyperparameters of HSIC-InfoGAN, and discuss the potential of HSIC-InfoGAN for medical applications.