Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
This addresses the problem of training instability in GANs for researchers and practitioners, offering incremental improvements by integrating semi-supervised learning techniques.
The paper tackles the training instability of Wasserstein GANs by introducing a novel consistency term to enforce Lipschitz continuity, resulting in improved photo-realistic samples and state-of-the-art semi-supervised learning performance, including an inception score over 5.0 with 1,000 CIFAR-10 images and over 90% accuracy with 4,000 labeled images.
Despite being impactful on a variety of problems and applications, the generative adversarial nets (GANs) are remarkably difficult to train. This issue is formally analyzed by \cite{arjovsky2017towards}, who also propose an alternative direction to avoid the caveats in the minmax two-player training of GANs. The corresponding algorithm, called Wasserstein GAN (WGAN), hinges on the 1-Lipschitz continuity of the discriminator. In this paper, we propose a novel approach to enforcing the Lipschitz continuity in the training procedure of WGANs. Our approach seamlessly connects WGAN with one of the recent semi-supervised learning methods. As a result, it gives rise to not only better photo-realistic samples than the previous methods but also state-of-the-art semi-supervised learning results. In particular, our approach gives rise to the inception score of more than 5.0 with only 1,000 CIFAR-10 images and is the first that exceeds the accuracy of 90% on the CIFAR-10 dataset using only 4,000 labeled images, to the best of our knowledge.