Generalization and Equilibrium in Generative Adversarial Nets (GANs)
It addresses generalization and equilibrium issues in GANs for machine learning practitioners, offering a theoretical framework and practical improvement, though incremental in nature.
The paper shows that GAN training may not generalize well to the target distribution in standard metrics, but does generalize under a weaker neural net distance, and proves an approximate pure equilibrium exists for a special generator class, leading to the MIX+GAN protocol that empirically improves existing GAN training.
We show that training of generative adversarial network (GAN) may not have good generalization properties; e.g., training may appear successful but the trained distribution may be far from target distribution in standard metrics. However, generalization does occur for a weaker metric called neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a special class of generators with natural training objectives when generator capacity and training set sizes are moderate. This existence of equilibrium inspires MIX+GAN protocol, which can be combined with any existing GAN training, and empirically shown to improve some of them.