Self-Supervised GAN to Counter Forgetting
This addresses training instability in GANs for machine learning practitioners, but it is incremental as it builds on prior work framing GAN training as continual learning.
The paper tackles the problem of forgetting in GAN discriminators during sequential training, which causes instability, by adding self-supervision to maintain useful representations without labels, closing the performance gap between conditional and unconditional models.
GANs involve training two networks in an adversarial game, where each network's task depends on its adversary. Recently, several works have framed GAN training as an online or continual learning problem. We focus on the discriminator, which must perform classification under an (adversarially) shifting data distribution. When trained on sequential tasks, neural networks exhibit \emph{forgetting}. For GANs, discriminator forgetting leads to training instability. To counter forgetting, we encourage the discriminator to maintain useful representations by adding a self-supervision. Conditional GANs have a similar effect using labels. However, our self-supervised GAN does not require labels, and closes the performance gap between conditional and unconditional models. We show that, in doing so, the self-supervised discriminator learns better representations than regular GANs.