Empirical Analysis of Overfitting and Mode Drop in GAN Training
This addresses issues in generative modeling for researchers, providing empirical insights that are incremental but clarify existing debates.
The study tackled overfitting and mode drop in GAN training by showing that removing stochasticity leads to overfitting with minimal mode drop, challenging the intuition that GANs do not memorize training data and that mode drop is due to objective properties rather than optimization.
We examine two key questions in GAN training, namely overfitting and mode drop, from an empirical perspective. We show that when stochasticity is removed from the training procedure, GANs can overfit and exhibit almost no mode drop. Our results shed light on important characteristics of the GAN training procedure. They also provide evidence against prevailing intuitions that GANs do not memorize the training set, and that mode dropping is mainly due to properties of the GAN objective rather than how it is optimized during training.