Asymmetric regularization mechanism for GAN training with Variational Inequalities
This addresses training instability for GAN users, but it appears incremental as it builds on existing regularization methods.
The authors tackled the instability of GAN training by formulating it as a Nash equilibrium problem and proposing an asymmetric regularization mechanism, achieving last-iterate linear convergence with explicit constants in simulations.
We formulate the training of generative adversarial networks (GANs) as a Nash equilibrium seeking problem. To stabilize the training process and find a Nash equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an equilibrium and stabilize the trajectory.