Adaptive Consensus Optimization Method for GANs
This work addresses the challenge of efficient and stable GAN training for image generation tasks, offering an incremental improvement over existing second-order methods.
The authors tackled the problem of training generative adversarial networks (GANs) by proposing a second-order gradient-based method that combines ADAM and RMSprop, which achieves faster convergence to similar accuracy compared to other second-order methods without requiring linear system solutions or additional derivative terms. The method produces better or comparable inception scores and image quality, validated on datasets like FFHQ, LSUN, CIFAR10, MNIST, and Fashion MNIST.
We propose a second order gradient based method with ADAM and RMSprop for the training of generative adversarial networks. The proposed method is fastest to obtain similar accuracy when compared to prominent second order methods. Unlike state-of-the-art recent methods, it does not require solving a linear system, or it does not require additional mixed second derivative terms. We derive the fixed point iteration corresponding to proposed method, and show that the proposed method is convergent. The proposed method produces better or comparable inception scores, and comparable quality of images compared to other recently proposed state-of-the-art second order methods. Compared to first order methods such as ADAM, it produces significantly better inception scores. The proposed method is compared and validated on popular datasets such as FFHQ, LSUN, CIFAR10, MNIST, and Fashion MNIST for image generation tasks\footnote{Accepted in IJCNN 2023}. Codes: \url{https://github.com/misterpawan/acom}