Wasserstein GAN Can Perform PCA
This provides theoretical insights into GAN optimality for researchers, though it is incremental as it focuses on a simplified setting.
The paper tackles the problem of understanding how far GANs are from optimality by analyzing them in a simple linear-generator Gaussian-data setting where the optimal generator performs PCA. It finds that the original GAN fails to recover the PCA solution, while Wasserstein GAN can approach it in the limit of sample size.
Generative Adversarial Networks (GANs) have become a powerful framework to learn generative models that arise across a wide variety of domains. While there has been a recent surge in the development of numerous GAN architectures with distinct optimization metrics, we are still lacking in our understanding on how far away such GANs are from optimality. In this paper, we make progress on a theoretical understanding of the GANs under a simple linear-generator Gaussian-data setting where the optimal maximum-likelihood generator is known to perform Principal Component Analysis (PCA). We find that the original GAN by Goodfellow et. al. fails to recover the optimal PCA solution. On the other hand, we show that Wasserstein GAN can approach the PCA solution in the limit of sample size, and hence it may serve as a basis for an optimal GAN architecture that yields the optimal generator for a wide range of data settings.