Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks
This provides theoretical insights into WGAN training dynamics, though it is incremental as it builds on existing WGAN analysis.
This paper analyzes Wasserstein GANs with GroupSort discriminators, showing that approximation error depends on network capacity and sample size, and finds that overly powerful generators can perform worse when discriminators are weak. Numerical experiments on Swiss roll and MNIST datasets confirm these theoretical findings.
Studied here are Wasserstein generative adversarial networks (WGANs) with GroupSort neural networks as their discriminators. It is shown that the error bound of the approximation for the target distribution depends on the width and depth (capacity) of the generators and discriminators and the number of samples in training. A quantified generalization bound is established for the Wasserstein distance between the generated and target distributions. According to the theoretical results, WGANs have a higher requirement for the capacity of discriminators than that of generators, which is consistent with some existing results. More importantly, the results with overly deep and wide (high-capacity) generators may be worse than those with low-capacity generators if discriminators are insufficiently strong. Numerical results obtained using Swiss roll and MNIST datasets confirm the theoretical results.