Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow
This work addresses training instability for researchers using particle-based distance GANs, offering an incremental improvement with a stabilizing method.
The paper tackles the unstable training problem in particle-based distance GANs like MMD GAN and Cramér GAN by analyzing stability from a probability density dynamics perspective, and proposes adding a stabilizing term to the discriminator loss to address this issue, with experimental validation.
In this paper, we investigate the training process of generative networks that use a type of probability density distance named particle-based distance as the objective function, e.g. MMD GAN, Cramér GAN, EIEG GAN. However, these GANs often suffer from the problem of unstable training. In this paper, we analyze the stability of the training process of these GANs from the perspective of probability density dynamics. In our framework, we regard the discriminator $D$ in these GANs as a feature transformation mapping that maps high dimensional data into a feature space, while the generator $G$ maps random variables to samples that resemble real data in terms of feature space. This perspective enables us to perform stability analysis for the training of GANs using the Wasserstein gradient flow of the probability density function. We find that the training process of the discriminator is usually unstable due to the formulation of $\min_G \max_D E(G, D)$ in GANs. To address this issue, we add a stabilizing term in the discriminator loss function. We conduct experiments to validate our stability analysis and stabilizing method.