Some Theoretical Properties of GANs
This work provides incremental theoretical insights into GANs, which are important for researchers in machine learning and statistics seeking to understand the foundations of generative models.
The paper tackles the theoretical understanding of Generative Adversarial Networks (GANs) by analyzing their mathematical and statistical properties, including connections to Jensen-Shannon divergence and large sample properties, with results illustrated through simulated examples.
Generative Adversarial Networks (GANs) are a class of generative algorithms that have been shown to produce state-of-the art samples, especially in the domain of image creation. The fundamental principle of GANs is to approximate the unknown distribution of a given data set by optimizing an objective function through an adversarial game between a family of generators and a family of discriminators. In this paper, we offer a better theoretical understanding of GANs by analyzing some of their mathematical and statistical properties. We study the deep connection between the adversarial principle underlying GANs and the Jensen-Shannon divergence, together with some optimality characteristics of the problem. An analysis of the role of the discriminator family via approximation arguments is also provided. In addition, taking a statistical point of view, we study the large sample properties of the estimated distribution and prove in particular a central limit theorem. Some of our results are illustrated with simulated examples.