A domain agnostic measure for monitoring and evaluating GANs
This addresses the need for a reliable evaluation metric for GANs across various domains, offering an incremental improvement over existing methods.
The paper tackles the problem of evaluating Generative Adversarial Networks (GANs) by proposing a domain-agnostic measure based on the duality gap from game theory, which effectively ranks models and monitors training progress, showing high correlation with FID on image datasets and domain-specific scores for text, sound, and cosmology data.
Generative Adversarial Networks (GANs) have shown remarkable results in modeling complex distributions, but their evaluation remains an unsettled issue. Evaluations are essential for: (i) relative assessment of different models and (ii) monitoring the progress of a single model throughout training. The latter cannot be determined by simply inspecting the generator and discriminator loss curves as they behave non-intuitively. We leverage the notion of duality gap from game theory to propose a measure that addresses both (i) and (ii) at a low computational cost. Extensive experiments show the effectiveness of this measure to rank different GAN models and capture the typical GAN failure scenarios, including mode collapse and non-convergent behaviours. This evaluation metric also provides meaningful monitoring on the progression of the loss during training. It highly correlates with FID on natural image datasets, and with domain specific scores for text, sound and cosmology data where FID is not directly suitable. In particular, our proposed metric requires no labels or a pretrained classifier, making it domain agnostic.