Adversarial Text Generation via Feature-Mover's Distance
This addresses the problem of mode collapse and training instability in text GANs for researchers in natural language processing, offering a more robust method.
The paper tackled the challenge of applying GANs to text generation by proposing a novel approach using optimal transport to match latent feature distributions, resulting in improved performance across tasks like unconditional text generation and style transfer.
Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text-generation tasks. Instead of using the standard GAN objective, we propose to improve text-generation GAN via a novel approach inspired by optimal transport. Specifically, we consider matching the latent feature distributions of real and synthetic sentences using a novel metric, termed the feature-mover's distance (FMD). This formulation leads to a highly discriminative critic and easy-to-optimize objective, overcoming the mode-collapsing and brittle-training problems in existing methods. Extensive experiments are conducted on a variety of tasks to evaluate the proposed model empirically, including unconditional text generation, style transfer from non-parallel text, and unsupervised cipher cracking. The proposed model yields superior performance, demonstrating wide applicability and effectiveness.