Sequential training of GANs against GAN-classifiers reveals correlated "knowledge gaps" present among independently trained GAN instances
This addresses the issue of out-of-distribution artifacts in GAN-generated images for researchers, but it is incremental as it builds on prior work on GAN-classifiers.
The study tackled the problem of 'knowledge gaps' in GANs by iteratively training GAN-classifiers and GANs to fool them, finding that StyleGAN2 could fool classifiers without quality loss, revealing an ordering over optima, while DCGAN could not.
Modern Generative Adversarial Networks (GANs) generate realistic images remarkably well. Previous work has demonstrated the feasibility of "GAN-classifiers" that are distinct from the co-trained discriminator, and operate on images generated from a frozen GAN. That such classifiers work at all affirms the existence of "knowledge gaps" (out-of-distribution artifacts across samples) present in GAN training. We iteratively train GAN-classifiers and train GANs that "fool" the classifiers (in an attempt to fill the knowledge gaps), and examine the effect on GAN training dynamics, output quality, and GAN-classifier generalization. We investigate two settings, a small DCGAN architecture trained on low dimensional images (MNIST), and StyleGAN2, a SOTA GAN architecture trained on high dimensional images (FFHQ). We find that the DCGAN is unable to effectively fool a held-out GAN-classifier without compromising the output quality. However, StyleGAN2 can fool held-out classifiers with no change in output quality, and this effect persists over multiple rounds of GAN/classifier training which appears to reveal an ordering over optima in the generator parameter space. Finally, we study different classifier architectures and show that the architecture of the GAN-classifier has a strong influence on the set of its learned artifacts.