Towards Efficient and Unbiased Implementation of Lipschitz Continuity in GANs
This addresses training stability and sample quality issues in GANs for machine learning practitioners, but it is incremental as it builds on existing Lipschitz continuity methods.
The paper tackled the problem of biased and restrictive implementations of Lipschitz continuity in GANs, proposing a new method that achieved successful training in situations where gradient penalty and spectral normalization failed.
Lipschitz continuity recently becomes popular in generative adversarial networks (GANs). It was observed that the Lipschitz regularized discriminator leads to improved training stability and sample quality. The mainstream implementations of Lipschitz continuity include gradient penalty and spectral normalization. In this paper, we demonstrate that gradient penalty introduces undesired bias, while spectral normalization might be over restrictive. We accordingly propose a new method which is efficient and unbiased. Our experiments verify our analysis and show that the proposed method is able to achieve successful training in various situations where gradient penalty and spectral normalization fail.