Learning Implicit Text Generation via Feature Matching
This work addresses text generation challenges for NLP researchers and practitioners, offering a stable alternative to adversarial methods, though it is incremental as it extends an existing image-based approach to sequential data.
The authors tackled the problem of generating sequential data like text by adapting generative feature matching networks (GFMN) from images to sequences, resulting in SeqGFMN, which outperforms adversarial methods in tasks such as unconditional text generation, class-conditional text generation, and unsupervised text style transfer.
Generative feature matching network (GFMN) is an approach for training implicit generative models for images by performing moment matching on features from pre-trained neural networks. In this paper, we present new GFMN formulations that are effective for sequential data. Our experimental results show the effectiveness of the proposed method, SeqGFMN, for three distinct generation tasks in English: unconditional text generation, class-conditional text generation, and unsupervised text style transfer. SeqGFMN is stable to train and outperforms various adversarial approaches for text generation and text style transfer.