Controllable Abstraction in Summary Generation for Large Language Models via Prompt Engineering
This work addresses the issue of summary quality and controllability for users of large language models, but it is incremental as it builds on existing prompt engineering techniques.
The study tackled the problem of generating controllable abstract summaries with large language models by developing a multi-stage prompt engineering framework, finding that prompt length significantly impacts summary quality and that data noise reduces ROUGE-L scores as it increases.
This study presents a controllable abstract summary generation method for large language models based on prompt engineering. To address the issues of summary quality and controllability in traditional methods, we design a multi-stage prompt generation framework. This framework generates summaries with varying levels of abstraction by performing semantic analysis, topic modeling, and noise control on the input text. The experiment uses the CNN/Daily Mail dataset and provides a detailed analysis of different prompt lengths, data noise, and text types. The experimental results show that prompt length has a significant impact on the quality of generated summaries. Both very short and very long prompt tokens result in a decrease in summary quality. Data noise also negatively affects the summary generation process. As noise levels increase, the ROUGE-L score gradually decreases. Furthermore, different text types have varying effects on the model's ability to generate summaries. The model performs best when handling news texts, while its performance is worse when processing academic articles. This research provides new insights into improving summary generation using large language models, particularly in how controlling prompt strategies and optimizing text preprocessing can enhance summary accuracy and controllability.