Local Stability and Performance of Simple Gradient Penalty mu-Wasserstein GAN
This work addresses training stability issues in Wasserstein GANs for machine learning practitioners, though it appears incremental as it builds on existing gradient penalty methods.
The paper proves local stability for a simple gradient penalty μ-WGAN under specific assumptions about equilibrium and penalty measures, using measure-valued differentiation to handle singular measures. Experimental results with unintuitive penalty measures support the theoretical findings.
Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance between a data distribution and sample distribution. Recent studies have proposed stabilizing the training process for the WGAN and implementing the Lipschitz constraint. In this study, we prove the local stability of optimizing the simple gradient penalty $μ$-WGAN(SGP $μ$-WGAN) under suitable assumptions regarding the equilibrium and penalty measure $μ$. The measure valued differentiation concept is employed to deal with the derivative of the penalty terms, which is helpful for handling abstract singular measures with lower dimensional support. Based on this analysis, we claim that penalizing the data manifold or sample manifold is the key to regularizing the original WGAN with a gradient penalty. Experimental results obtained with unintuitive penalty measures that satisfy our assumptions are also provided to support our theoretical results.