A method for escaping limit cycles in training GANs
This addresses training instability in GANs for machine learning practitioners, but it is incremental as it builds on existing methods like Adam.
The paper tackles the problem of limit cycling behavior in training GANs by proposing the predictive centripetal acceleration algorithm (PCAA), which improves last-iterate convergence bounds and is validated on datasets like CelebA.
This paper mainly conducts further research to alleviate the issue of limit cycling behavior in training generative adversarial networks (GANs) through the proposed predictive centripetal acceleration algorithm (PCAA). Specifically, we first derive the upper and lower bounds on the last-iterate convergence rates of PCAA for the general bilinear game, with the upper bound notably improving upon previous results. Then, we combine PCAA with the adaptive moment estimation algorithm (Adam) to propose PCAA-Adam, a practical approach for training GANs. Finally, we validate the effectiveness of the proposed algorithm through experiments conducted on bilinear games, multivariate Gaussian distributions, and the CelebA dataset, respectively.