Cooperate or Compete: A New Perspective on Training of Generative Networks
This addresses a key bottleneck in GAN training for researchers and practitioners, offering a novel perspective to enhance stability and efficiency.
The paper tackles the training instability in Generative Adversarial Networks (GANs) by proposing a cooperative approach modeled as an infinitely repeated game, resulting in faster learning and improved performance for both generator and discriminator modules.
GANs have two competing modules: the generator module is trained to generate new examples, and the discriminator module is trained to discriminate real examples from generated examples. The training procedure of GAN is modeled as a finitely repeated simultaneous game. Each module tries to increase its performance at every repetition of the base game (at every batch of training data) in a non-cooperative manner. We observed that each module can perform better and learn faster if training is modeled as an infinitely repeated simultaneous game. At every repetition of the base game (at every batch of training data) the stronger module (whose performance is increased or remains the same compared to the previous batch of training data) cooperates with the weaker module (whose performance is decreased compared to the previous batch of training data) and only the weaker module is allowed to increase its performance.