PAC-Bayesian Generalization Bounds for Adversarial Generative Models
This work provides theoretical generalization guarantees for generative models, which is incremental but addresses a key issue in machine learning for researchers and practitioners using GANs.
The paper tackles the problem of generalization in adversarial generative models by extending PAC-Bayesian theory to develop bounds based on Wasserstein and total variation distances, with results applied to Wasserstein GANs and Energy-Based GANs, showing non-vacuous bounds in synthetic experiments.
We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.