On Convergence and Stability of GANs
This work addresses the stability and convergence issues in GANs, which are crucial for generating high-quality and diverse data in machine learning applications, representing a novel method for a known bottleneck.
The paper tackles the problem of mode collapse in GAN training by analyzing it as regret minimization, identifying undesirable local equilibria as the cause, and proposes DRAGAN, a gradient penalty scheme that avoids these equilibria, resulting in faster training, improved stability with fewer mode collapses, and better modeling performance across various architectures and objective functions.
We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse. We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points. We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions.