Instruct-SCTG: Guiding Sequential Controlled Text Generation through Instructions
This addresses the challenge of generating structurally coherent text for applications requiring organized discourse, though it appears incremental as it builds on existing instruction-tuning methods.
The paper tackles the problem of maintaining human-like discourse structure in text generated by instruction-tuned language models, proposing Instruct-SCTG, a sequential framework that generates articles section-by-section using natural language instructions. The result is state-of-the-art performance on three datasets from news and recipe domains, as verified by both automatic and human evaluation.
Instruction-tuned large language models have shown remarkable performance in aligning generated text with user intentions across various tasks. However, maintaining human-like discourse structure in the generated text remains a challenging research question. In this paper, we propose Instruct-SCTG, a flexible and effective sequential framework that harnesses instruction-tuned language models to generate structurally coherent text in both fine-tuned and zero-shot setups. Our framework generates articles in a section-by-section manner, aligned with the desired human structure using natural language instructions. Furthermore, we introduce a new automatic metric that measures discourse divergence in a fuzzy manner. Extensive experiments on three datasets from representative domains of news and recipes demonstrate the state-of-the-art performance of our framework in imposing discourse structure during text generation, as verified by both automatic and human evaluation. Our code will be available on Github.