Mimicry: Towards the Reproducibility of GAN Research
This addresses reproducibility issues for researchers in generative modeling, though it is incremental as it builds on existing GAN methods without proposing new ones.
The paper tackles the problem of inconsistent implementations and evaluations in GAN research by introducing Mimicry, a PyTorch library that provides standardized implementations of popular GANs and metrics, resulting in reproducible baseline performances across seven datasets and three metrics.
Advancing the state of Generative Adversarial Networks (GANs) research requires one to make careful and accurate comparisons with existing works. Yet, this is often difficult to achieve in practice when models are often implemented differently using varying frameworks, and evaluated using different procedures even when the same metric is used. To mitigate these issues, we introduce Mimicry, a lightweight PyTorch library that provides implementations of popular state-of-the-art GANs and evaluation metrics to closely reproduce reported scores in the literature. We provide comprehensive baseline performances of different GANs on seven widely-used datasets by training these GANs under the same conditions, and evaluating them across three popular GAN metrics using the same procedures. The library can be found at https://github.com/kwotsin/mimicry.