GMM-Based Generative Adversarial Encoder Learning
This work provides a generic framework for integrating an encoder into existing GAN architectures, improving both latent space learning for clustering and image generation quality for researchers and practitioners working with GANs.
This paper addresses the limitation of Generative Adversarial Networks (GANs) in directly inferring a latent space by integrating an encoder with the discriminator using shared weights and a novel loss term. The proposed method models the encoder's latent space with a Gaussian Mixture Model (GMM), resulting in improved image generation (better IS and FID scores for Vanilla GAN and Wasserstein GAN) and competitive clustering performance.
While GAN is a powerful model for generating images, its inability to infer a latent space directly limits its use in applications requiring an encoder. Our paper presents a simple architectural setup that combines the generative capabilities of GAN with an encoder. We accomplish this by combining the encoder with the discriminator using shared weights, then training them simultaneously using a new loss term. We model the output of the encoder latent space via a GMM, which leads to both good clustering using this latent space and improved image generation by the GAN. Our framework is generic and can be easily plugged into any GAN strategy. In particular, we demonstrate it both with Vanilla GAN and Wasserstein GAN, where in both it leads to an improvement in the generated images in terms of both the IS and FID scores. Moreover, we show that our encoder learns a meaningful representation as its clustering results are competitive with the current GAN-based state-of-the-art in clustering.