Variance Regularizing Adversarial Learning
This addresses a fundamental issue in GAN training for image generation, though it appears incremental as it modifies existing adversarial frameworks rather than introducing a new paradigm.
The paper tackles the problem of vanishing gradients in adversarial training by replacing the discriminator score with a bi-modal Gaussian distribution, which ensures non-zero gradients even with a perfect classifier. The method was tested on standard benchmark image datasets and showed smooth classifier output distributions with overlap between real and fake modes.
We introduce a novel approach for training adversarial models by replacing the discriminator score with a bi-modal Gaussian distribution over the real/fake indicator variables. In order to do this, we train the Gaussian classifier to match the target bi-modal distribution implicitly through meta-adversarial training. We hypothesize that this approach ensures a non-zero gradient to the generator, even in the limit of a perfect classifier. We test our method against standard benchmark image datasets as well as show the classifier output distribution is smooth and has overlap between the real and fake modes.