A Unified View of cGANs with and without Classifiers
This work addresses the challenge of designing better cGANs for generating high-quality class-conditional samples, particularly in complex domains like ImageNet, though it is incremental as it builds on existing cGAN variants.
The paper tackles the problem of improving conditional Generative Adversarial Networks (cGANs) by leveraging classifiers, which were previously avoided due to limitations like generating only easy-to-classify samples. It proposes a unified framework that connects cGANs and classification, outperforming state-of-the-art cGANs on benchmark datasets, including a 5% improvement in FID on ImageNet.
Conditional Generative Adversarial Networks (cGANs) are implicit generative models which allow to sample from class-conditional distributions. Existing cGANs are based on a wide range of different discriminator designs and training objectives. One popular design in earlier works is to include a classifier during training with the assumption that good classifiers can help eliminate samples generated with wrong classes. Nevertheless, including classifiers in cGANs often comes with a side effect of only generating easy-to-classify samples. Recently, some representative cGANs avoid the shortcoming and reach state-of-the-art performance without having classifiers. Somehow it remains unanswered whether the classifiers can be resurrected to design better cGANs. In this work, we demonstrate that classifiers can be properly leveraged to improve cGANs. We start by using the decomposition of the joint probability distribution to connect the goals of cGANs and classification as a unified framework. The framework, along with a classic energy model to parameterize distributions, justifies the use of classifiers for cGANs in a principled manner. It explains several popular cGAN variants, such as ACGAN, ProjGAN, and ContraGAN, as special cases with different levels of approximations, which provides a unified view and brings new insights to understanding cGANs. Experimental results demonstrate that the design inspired by the proposed framework outperforms state-of-the-art cGANs on multiple benchmark datasets, especially on the most challenging ImageNet. The code is available at https://github.com/sian-chen/PyTorch-ECGAN.