Personalized Abstractive Summarization by Tri-agent Generation Pipeline
This addresses the problem of output personalization for users of large language models, but it is incremental as it builds on existing models with a novel pipeline.
The paper tackles the challenge of personalizing outputs from large language models like ChatGPT to user preferences by proposing a tri-agent generation pipeline with a generator, instructor, and editor, and shows it effectively generates outputs that better meet user expectations on two abstractive summarization datasets.
Tailoring outputs from large language models, like ChatGPT, to implicit user preferences remains a challenge despite their impressive generative capabilities. In this paper, we propose a tri-agent generation pipeline comprising a generator, an instructor, and an editor to enhance output personalization. The generator produces an initial output, the instructor automatically generates editing instructions based on user preferences, and the editor refines the output to align with those preferences. The inference-only large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation. We train the instructor using editor-steered reinforcement learning, leveraging feedback from a large-scale editor model to optimize instruction generation. Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better meet user expectations. Code is available at \url{https://github.com/Wendy-Xiao/chatgpt_editing_summ}