AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation
This work addresses the challenge of producing high-quality text under strict lexical constraints, which is useful for applications like content creation and summarization, but it is incremental as it builds on prior constrained generation methods.
The paper tackles the problem of generating text that must include specific words, where existing methods trade off quality for constraint satisfaction. The authors introduce AutoTemplate, a framework that first generates text with placeholders and then fills them with required words, achieving better performance than baselines on keywords-to-sentence and entity-guided summarization tasks while meeting all constraints.
Lexically constrained text generation is one of the constrained text generation tasks, which aims to generate text that covers all the given constraint lexicons. While the existing approaches tackle this problem using a lexically constrained beam search algorithm or dedicated model using non-autoregressive decoding, there is a trade-off between the generated text quality and the hard constraint satisfaction. We introduce AutoTemplate, a simple yet effective lexically constrained text generation framework divided into template generation and lexicalization tasks. The template generation is to generate the text with the placeholders, and lexicalization replaces them into the constraint lexicons to perform lexically constrained text generation. We conducted the experiments on two tasks: keywords-to-sentence generations and entity-guided summarization. Experimental results show that the AutoTemplate outperforms the competitive baselines on both tasks while satisfying the hard lexical constraints. The code is available at https://github.com/megagonlabs/autotemplate