A Classification-Based Study of Covariate Shift in GAN Distributions
This addresses a fundamental gap in GAN evaluation for researchers and practitioners, though it is incremental as it builds on prior work with new tools.
The paper tackles the problem of evaluating whether GANs capture all distributional characteristics of training data, particularly diversity, by developing quantitative tools to assess covariate shift like mode collapse and boundary distortion, finding that popular GANs have significant issues in reproducing distributional properties.
A basic, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether they are truly able to capture all the fundamental characteristics of the distributions they are trained on. In particular, evaluating the diversity of GAN distributions is challenging and existing methods provide only a partial understanding of this issue. In this paper, we develop quantitative and scalable tools for assessing the diversity of GAN distributions. Specifically, we take a classification-based perspective and view loss of diversity as a form of covariate shift introduced by GANs. We examine two specific forms of such shift: mode collapse and boundary distortion. In contrast to prior work, our methods need only minimal human supervision and can be readily applied to state-of-the-art GANs on large, canonical datasets. Examining popular GANs using our tools indicates that these GANs have significant problems in reproducing the more distributional properties of their training dataset.