Approximation and Convergence Properties of Generative Adversarial Learning
This work addresses foundational theoretical gaps in GANs, which are crucial for improving their reliability and performance in generative modeling tasks across machine learning.
The paper tackles the problem of understanding how well generative adversarial networks (GANs) approximate target data distributions by analyzing the effects of restricted discriminator families and objective functions on approximation quality and convergence. It shows that under certain conditions, restricted discriminators lead to moment-matching and that strict adversarial divergences ensure convergence in the objective implies weak convergence to the target distribution.
Generative adversarial networks (GAN) approximate a target data distribution by jointly optimizing an objective function through a "two-player game" between a generator and a discriminator. Despite their empirical success, however, two very basic questions on how well they can approximate the target distribution remain unanswered. First, it is not known how restricting the discriminator family affects the approximation quality. Second, while a number of different objective functions have been proposed, we do not understand when convergence to the global minima of the objective function leads to convergence to the target distribution under various notions of distributional convergence. In this paper, we address these questions in a broad and unified setting by defining a notion of adversarial divergences that includes a number of recently proposed objective functions. We show that if the objective function is an adversarial divergence with some additional conditions, then using a restricted discriminator family has a moment-matching effect. Additionally, we show that for objective functions that are strict adversarial divergences, convergence in the objective function implies weak convergence, thus generalizing previous results.