Penalty Gradient Normalization for Generative Adversarial Networks
This addresses instability issues in GAN training for researchers and practitioners, offering an incremental improvement over existing normalization methods.
The paper tackles training instability in Generative Adversarial Networks (GANs) by proposing penalty gradient normalization (PGN), which constrains the discriminator's gradient norm to improve performance, resulting in better Frechet Inception Distance and Inception Score on three datasets.
In this paper, we propose a novel normalization method called penalty gradient normalization (PGN) to tackle the training instability of Generative Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing work such as gradient penalty and spectral normalization, the proposed PGN only imposes a penalty gradient norm constraint on the discriminator function, which increases the capacity of the discriminator. Moreover, the proposed penalty gradient normalization can be applied to different GAN architectures with little modification. Extensive experiments on three datasets show that GANs trained with penalty gradient normalization outperform existing methods in terms of both Frechet Inception and Distance and Inception Score.