Quantitatively Evaluating GANs With Divergences Proposed for Training
This addresses the problem of model assessment for researchers and practitioners in generative modeling, though it is incremental as it applies existing divergence functions to evaluation rather than proposing new ones.
The paper tackles the lack of quantitative methods for evaluating GANs by using divergence and distance functions typically reserved for training to assess various GAN variants, finding consistency across metrics and that test-time metrics do not favor networks using the same training criterion.
Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application. However, we currently lack quantitative methods for model assessment. Because of this, while many GAN variants are being proposed, we have relatively little understanding of their relative abilities. In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training. We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion. We also compare the proposed metrics to human perceptual scores.