Consistency Regularization for Generative Adversarial Networks
This work addresses the challenge of unstable training in GANs, a key issue for researchers and practitioners in generative modeling, though it is incremental as it adapts an existing technique from semi-supervised learning.
The paper tackles the problem of stabilizing GAN training by proposing consistency regularization, which penalizes discriminator sensitivity to data augmentations, achieving state-of-the-art FID scores such as 11.48 on CIFAR-10 and 6.66 on ImageNet-2012 for conditional generation.
Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization---a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of the discriminator to these augmentations. We conduct a series of experiments to demonstrate that consistency regularization works effectively with spectral normalization and various GAN architectures, loss functions and optimizer settings. Our method achieves the best FID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Moreover, Our consistency regularized GAN (CR-GAN) improves state-of-the-art FID scores for conditional generation from 14.73 to 11.48 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012.