Gradient Normalization for Generative Adversarial Networks
This addresses instability issues in GAN training for researchers and practitioners, offering a method that can be applied to various architectures with minimal modification, though it appears incremental as it builds on prior normalization techniques.
The paper tackles training instability in Generative Adversarial Networks (GANs) by proposing gradient normalization (GN), which imposes a hard 1-Lipschitz constraint on the discriminator to increase its capacity, and experiments on four datasets show it outperforms existing methods in Frechet Inception Distance and Inception Score.
In this paper, we propose a novel normalization method called gradient normalization (GN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed GN only imposes a hard 1-Lipschitz constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on four datasets show that GANs trained with gradient normalization outperform existing methods in terms of both Frechet Inception Distance and Inception Score.