Alleviation of Gradient Exploding in GANs: Fake Can Be Real
This addresses instability in GAN training, a key issue for researchers and practitioners in generative modeling, though it is an incremental improvement over existing methods.
The paper tackles the mode collapse problem in GANs by proposing a training method where certain fake samples are treated as real, which reduces gradient exploding and stabilizes training, resulting in a more faithful generated distribution as verified on various datasets.
In order to alleviate the notorious mode collapse phenomenon in generative adversarial networks (GANs), we propose a novel training method of GANs in which certain fake samples are considered as real ones during the training process. This strategy can reduce the gradient value that generator receives in the region where gradient exploding happens. We show the process of an unbalanced generation and a vicious circle issue resulted from gradient exploding in practical training, which explains the instability of GANs. We also theoretically prove that gradient exploding can be alleviated by penalizing the difference between discriminator outputs and fake-as-real consideration for very close real and fake samples. Accordingly, Fake-As-Real GAN (FARGAN) is proposed with a more stable training process and a more faithful generated distribution. Experiments on different datasets verify our theoretical analysis.