DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation
This addresses the challenge of producing coherent and content-rich opinion texts for applications like argumentation and journalism, representing an incremental advance in text generation methods.
The paper tackles the problem of generating coherent long-form opinion text by proposing DYPLOC, a framework that uses dynamic content planning with mixed language models, and it shows significant improvements in automatic evaluations and human judgments for tasks like argument generation and opinion article writing.
We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times' Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.