A Large-Scale Study on Regularization and Normalization in GANs
This work addresses reproducibility and practical training issues for researchers and practitioners using GANs, but it is incremental as it synthesizes existing methods rather than introducing new ones.
The paper tackles the challenge of training GANs, which is difficult due to hyperparameter tuning and lack of failure mode quantification, by conducting a large-scale study to evaluate common pitfalls and reproducibility issues, providing open-source code and pre-trained models.
Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of "tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.