An Online Learning Approach to Generative Adversarial Networks
This addresses the training difficulties in GANs, which are crucial for generative modeling in machine learning, but the method is incremental as it builds on existing online learning ideas.
The paper tackles the instability in training Generative Adversarial Networks (GANs) by proposing a novel online learning method called Chekhov GAN, which improves stability and performance on real-world tasks compared to standard training.
We consider the problem of training generative models with a Generative Adversarial Network (GAN). Although GANs can accurately model complex distributions, they are known to be difficult to train due to instabilities caused by a difficult minimax optimization problem. In this paper, we view the problem of training GANs as finding a mixed strategy in a zero-sum game. Building on ideas from online learning we propose a novel training method named Chekhov GAN 1 . On the theory side, we show that our method provably converges to an equilibrium for semi-shallow GAN architectures, i.e. architectures where the discriminator is a one layer network and the generator is arbitrary. On the practical side, we develop an efficient heuristic guided by our theoretical results, which we apply to commonly used deep GAN architectures. On several real world tasks our approach exhibits improved stability and performance compared to standard GAN training.