Sentence-Level Content Planning and Style Specification for Neural Text Generation
This work addresses the challenge of generating coherent and style-appropriate text for tasks like persuasive arguments, Wikipedia paragraphs, and scientific abstracts, representing an incremental advance in neural text generation.
The paper tackles the problem of incoherent and unfaithful outputs in neural text generation by introducing a two-step model with sentence-level content planning and style specification, achieving significant improvements in automatic evaluations and higher human ratings for fluency and correctness compared to variants without style consideration.
Building effective text generation systems requires three critical components: content selection, text planning, and surface realization, and traditionally they are tackled as separate problems. Recent all-in-one style neural generation models have made impressive progress, yet they often produce outputs that are incoherent and unfaithful to the input. To address these issues, we present an end-to-end trained two-step generation model, where a sentence-level content planner first decides on the keyphrases to cover as well as a desired language style, followed by a surface realization decoder that generates relevant and coherent text. For experiments, we consider three tasks from domains with diverse topics and varying language styles: persuasive argument construction from Reddit, paragraph generation for normal and simple versions of Wikipedia, and abstract generation for scientific articles. Automatic evaluation shows that our system can significantly outperform competitive comparisons. Human judges further rate our system generated text as more fluent and correct, compared to the generations by its variants that do not consider language style.