Statistical Error Bounds for GANs with Nonlinear Objective Functionals
This work addresses the need for theoretical guarantees in GANs beyond linear metrics, which is incremental but important for researchers in machine learning and statistics.
The paper tackles the problem of deriving statistical error bounds for a broad class of GANs with nonlinear objective functionals, known as $(f,\Gamma)$-GANs, and proves their statistical consistency with finite-sample concentration inequalities, reducing to known results for IPM-GANs in the limit.
Generative adversarial networks (GANs) are unsupervised learning methods for training a generator distribution to produce samples that approximate those drawn from a target distribution. Many such methods can be formulated as minimization of a metric or divergence between probability distributions. Recent works have derived statistical error bounds for GANs that are based on integral probability metrics (IPMs), e.g., WGAN which is based on the 1-Wasserstein metric. In general, IPMs are defined by optimizing a linear functional (difference of expectations) over a space of discriminators. A much larger class of GANs, which we here call $(f,Γ)$-GANs, can be constructed using $f$-divergences (e.g., Jensen-Shannon, KL, or $α$-divergences) together with a regularizing discriminator space $Γ$ (e.g., $1$-Lipschitz functions). These GANs have nonlinear objective functions, depending on the choice of $f$, and have been shown to exhibit improved performance in a number of applications. In this work we derive statistical error bounds for $(f,Γ)$-GANs for general classes of $f$ and $Γ$ in the form of finite-sample concentration inequalities. These results prove the statistical consistency of $(f,Γ)$-GANs and reduce to the known results for IPM-GANs in the appropriate limit. Our results use novel Rademacher complexity bounds which provide new insight into the performance of IPM-GANs for distributions with unbounded support and have application to statistical learning tasks beyond GANs.