Lipschitz Constrained GANs via Boundedness and Continuity
This addresses training stability issues in GANs for machine learning practitioners, offering a theoretically grounded and efficient solution, though it is incremental as it builds on existing Lipschitz constraint methods.
The paper tackles the challenge of controlling GAN performance by introducing boundedness and continuity (BC) conditions to enforce Lipschitz constraints on discriminators, proving theoretical satisfaction and showing that BC-GANs achieve better performance and lower computational complexity compared to methods like gradient penalty and spectral normalization.
One of the challenges in the study of Generative Adversarial Networks (GANs) is the difficulty of its performance control. Lipschitz constraint is essential in guaranteeing training stability for GANs. Although heuristic methods such as weight clipping, gradient penalty and spectral normalization have been proposed to enforce Lipschitz constraint, it is still difficult to achieve a solution that is both practically effective and theoretically provably satisfying a Lipschitz constraint. In this paper, we introduce the boundedness and continuity ($BC$) conditions to enforce the Lipschitz constraint on the discriminator functions of GANs. We prove theoretically that GANs with discriminators meeting the BC conditions satisfy the Lipschitz constraint. We present a practically very effective implementation of a GAN based on a convolutional neural network (CNN) by forcing the CNN to satisfy the $BC$ conditions (BC-GAN). We show that as compared to recent techniques including gradient penalty and spectral normalization, BC-GANs not only have better performances but also lower computational complexity.