Sentence-Permuted Paragraph Generation
This work addresses the problem of low content diversity in paragraph generation for applications requiring varied outputs, representing an incremental improvement over prior methods.
The paper tackles the problem of generating diverse paragraphs by addressing the fixed left-to-right sentence order in existing models, proposing a framework that permutes sentence orders to improve content diversity, and demonstrates on three benchmarks that it generates more diverse outputs with higher quality than existing models.
Generating paragraphs of diverse contents is important in many applications. Existing generation models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. Our idea is permuting the sentence orders to improve the content diversity of multi-sentence paragraph. We propose a novel framework PermGen whose objective is to maximize the expected log-likelihood of output paragraph distributions with respect to all possible sentence orders. PermGen uses hierarchical positional embedding and designs new procedures for training, decoding, and candidate ranking in the sentence-permuted generation. Experiments on three paragraph generation benchmarks demonstrate PermGen generates more diverse outputs with a higher quality than existing models.