Generative Cooperative Networks for Natural Language Generation
This addresses a key bottleneck in applying GANs to discrete outputs like language, offering a more stable method for researchers and practitioners in NLG.
The paper tackles the instability of GANs in natural language generation by introducing Generative Cooperative Networks, which use a cooperative discriminator to improve text generation, achieving state-of-the-art results in two NLG tasks.
Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.