Stabilizing GAN Training with Multiple Random Projections
This addresses the problem of training instability in GANs for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the instability of GAN training in high dimensions by training a single generator against multiple discriminators, each using random low-dimensional projections, which prevents discriminators from perfectly rejecting samples and provides continuous gradients. This approach produces higher-quality image samples compared to traditional single-discriminator training.
Training generative adversarial networks is unstable in high-dimensions as the true data distribution tends to be concentrated in a small fraction of the ambient space. The discriminator is then quickly able to classify nearly all generated samples as fake, leaving the generator without meaningful gradients and causing it to deteriorate after a point in training. In this work, we propose training a single generator simultaneously against an array of discriminators, each of which looks at a different random low-dimensional projection of the data. Individual discriminators, now provided with restricted views of the input, are unable to reject generated samples perfectly and continue to provide meaningful gradients to the generator throughout training. Meanwhile, the generator learns to produce samples consistent with the full data distribution to satisfy all discriminators simultaneously. We demonstrate the practical utility of this approach experimentally, and show that it is able to produce image samples with higher quality than traditional training with a single discriminator.