WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points
This provides theoretical guarantees for GAN stability, addressing a key issue for researchers in deep generative models, though it is incremental as it builds on prior work on wide networks.
The paper tackles the problem of spurious stationary points in GAN training by proving that with an infinitely wide generator and finite-width discriminator, there are no such points, and this extends to arbitrarily large regions as width increases.
Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.