Training Generative Adversarial Networks via stochastic Nash games
This work addresses the training instability in GANs, a key issue for researchers and practitioners in generative modeling, by providing a theoretically grounded algorithm with convergence guarantees, though it is incremental as it builds on existing Nash game frameworks.
The paper tackles the challenge of training Generative Adversarial Networks (GANs) by modeling it as a stochastic Nash equilibrium problem and proposes a stochastic relaxed forward-backward (SRFB) algorithm, showing convergence to an exact solution with increasing data and to a neighborhood with few samples under weak monotonicity assumptions, applied to image generation.
Generative adversarial networks (GANs) are a class of generative models with two antagonistic neural networks: a generator and a discriminator. These two neural networks compete against each other through an adversarial process that can be modeled as a stochastic Nash equilibrium problem. Since the associated training process is challenging, it is fundamental to design reliable algorithms to compute an equilibrium. In this paper, we propose a stochastic relaxed forward-backward (SRFB) algorithm for GANs and we show convergence to an exact solution when an increasing number of data is available. We also show convergence of an averaged variant of the SRFB algorithm to a neighborhood of the solution when only few samples are available. In both cases, convergence is guaranteed when the pseudogradient mapping of the game is monotone. This assumption is among the weakest known in the literature. Moreover, we apply our algorithm to the image generation problem.