Dualing GANs
This addresses training instability for GAN users, but it is incremental as it builds on existing GAN frameworks.
The paper tackles the instability problem in GAN training by dualizing the discriminator, reformulating the objective into a maximization problem that stabilizes training for linear discriminators and offers an alternative algorithm for nonlinear ones.
Generative adversarial nets (GANs) are a promising technique for modeling a distribution from samples. It is however well known that GAN training suffers from instability due to the nature of its maximin formulation. In this paper, we explore ways to tackle the instability problem by dualizing the discriminator. We start from linear discriminators in which case conjugate duality provides a mechanism to reformulate the saddle point objective into a maximization problem, such that both the generator and the discriminator of this 'dualing GAN' act in concert. We then demonstrate how to extend this intuition to non-linear formulations. For GANs with linear discriminators our approach is able to remove the instability in training, while for GANs with nonlinear discriminators our approach provides an alternative to the commonly used GAN training algorithm.