GM Score: Incorporating inter-class and intra-class generator diversity, discriminability of disentangled representation, and sample fidelity for evaluating GANs
This addresses the problem of comprehensive GAN evaluation for researchers, though it is incremental as it builds on existing metrics.
The authors tackled the challenge of evaluating GANs by proposing GM Score, a new metric that incorporates sample quality, disentangled representation, and diversity, and they tested it on various GAN models trained on MNIST, showing improved evaluation capabilities.
While generative adversarial networks (GAN) are popular for their higher sample quality as opposed to other generative models like the variational autoencoders (VAE) and Boltzmann machines, they suffer from the same difficulty of the evaluation of generated samples. Various aspects must be kept in mind, such as the quality of generated samples, the diversity of classes (within a class and among classes), the use of disentangled latent spaces, agreement of said evaluation metric with human perception, etc. In this paper, we propose a new score, namely, GM Score, which takes into various factors such as sample quality, disentangled representation, intra-class and inter-class diversity, and other metrics such as precision, recall, and F1 score are employed for discriminability of latent space of deep belief network (DBN) and restricted Boltzmann machine (RBM). The evaluation is done for different GANs (GAN, DCGAN, BiGAN, CGAN, CoupledGAN, LSGAN, SGAN, WGAN, and WGAN Improved) trained on the benchmark MNIST dataset.