Finding Mixed Nash Equilibria of Generative Adversarial Networks
This addresses the longstanding issue of provable convergence in GAN training for machine learning researchers and practitioners, offering a more stable and efficient approach.
The paper tackles the problem of training Generative Adversarial Networks (GANs) by developing a novel algorithmic framework based on mixed Nash Equilibria, proving rigorous convergence rates and showing experimentally that it outperforms existing methods like SGD, Adam, and RMSProp in speed and quality.
We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective. Inspired by the classical prox methods, we develop a novel algorithmic framework for GANs via an infinite-dimensional two-player game and prove rigorous convergence rates to the mixed NE, resolving the longstanding problem that no provably convergent algorithm exists for general GANs. We then propose a principled procedure to reduce our novel prox methods to simple sampling routines, leading to practically efficient algorithms. Finally, we provide experimental evidence that our approach outperforms methods that seek pure strategy equilibria, such as SGD, Adam, and RMSProp, both in speed and quality.