Training GANs with Centripetal Acceleration
This addresses training instability in GANs, but appears incremental as it builds on existing gradient-based methods.
The paper tackles the problem of cyclic behaviors in GAN training by introducing centripetal acceleration methods, showing linear convergence for bilinear games and demonstrating superiority in numerical experiments.
Training generative adversarial networks (GANs) often suffers from cyclic behaviors of iterates. Based on a simple intuition that the direction of centripetal acceleration of an object moving in uniform circular motion is toward the center of the circle, we present the Simultaneous Centripetal Acceleration (SCA) method and the Alternating Centripetal Acceleration (ACA) method to alleviate the cyclic behaviors. Under suitable conditions, gradient descent methods with either SCA or ACA are shown to be linearly convergent for bilinear games. Numerical experiments are conducted by applying ACA to existing gradient-based algorithms in a GAN setup scenario, which demonstrate the superiority of ACA.