Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View
This work addresses a key bottleneck in disentangled representation learning for machine learning researchers, offering a method to improve disentanglement without sacrificing generation quality, though it is incremental as it builds on existing generative models.
The paper tackles the trade-off between disentanglement and generation quality in representation learning by leveraging pretrained generative models and discovering traversal directions as factors, achieving state-of-the-art results in disentangled representation learning across GAN, VAE, and Flow models.
From the intuitive notion of disentanglement, the image variations corresponding to different factors should be distinct from each other, and the disentangled representation should reflect those variations with separate dimensions. To discover the factors and learn disentangled representation, previous methods typically leverage an extra regularization term when learning to generate realistic images. However, the term usually results in a trade-off between disentanglement and generation quality. For the generative models pretrained without any disentanglement term, the generated images show semantically meaningful variations when traversing along different directions in the latent space. Based on this observation, we argue that it is possible to mitigate the trade-off by $(i)$ leveraging the pretrained generative models with high generation quality, $(ii)$ focusing on discovering the traversal directions as factors for disentangled representation learning. To achieve this, we propose Disentaglement via Contrast (DisCo) as a framework to model the variations based on the target disentangled representations, and contrast the variations to jointly discover disentangled directions and learn disentangled representations. DisCo achieves the state-of-the-art disentangled representation learning and distinct direction discovering, given pretrained non-disentangled generative models including GAN, VAE, and Flow. Source code is at https://github.com/xrenaa/DisCo.