Exploiting the Hidden Tasks of GANs: Making Implicit Subproblems Explicit
This work addresses training instability in GANs for generative modeling, offering a novel perspective that could enhance performance, but it appears incremental as it builds on existing GAN frameworks.
The authors tackled the problem of GAN training by revealing that the generator update decomposes into two implicit subproblems: generating inverse examples from the discriminator and using them for regression. They demonstrated significant improvements over standard GAN training, though no concrete numbers were provided.
We present an alternative perspective on the training of generative adversarial networks (GANs), showing that the training step for a GAN generator decomposes into two implicit subproblems. In the first, the discriminator provides new target data to the generator in the form of "inverse examples" produced by approximately inverting classifier labels. In the second, these examples are used as targets to update the generator via least-squares regression, regardless of the main loss specified to train the network. We experimentally validate our main theoretical result and demonstrate significant improvements over standard GAN training made possible by making these subproblems explicit.