Controllable Topic-Focused Abstractive Summarization
This work addresses the need for customizable summaries tailored to user-defined topics, offering an incremental improvement in topic-focused abstractive summarization.
The paper tackles the problem of generating abstractive summaries focused on specific topics by introducing a Transformer-based architecture that modifies cross-attention for topic control without adding parameters. It achieves state-of-the-art results on the NEWTS dataset and improves performance on CNN/Dailymail and XSum benchmarks, with human evaluation showing it generates more faithful summaries than the Frost model.
Controlled abstractive summarization focuses on producing condensed versions of a source article to cover specific aspects by shifting the distribution of generated text towards a desired style, e.g., a set of topics. Subsequently, the resulting summaries may be tailored to user-defined requirements. This paper presents a new Transformer-based architecture capable of producing topic-focused summaries. The architecture modifies the cross-attention mechanism of the Transformer to bring topic-focus control to the generation process while not adding any further parameters to the model. We show that our model sets a new state of the art on the NEWTS dataset in terms of topic-focused abstractive summarization as well as a topic-prevalence score. Moreover, we show via extensive experiments that our proposed topical cross-attention mechanism can be plugged into various Transformer models, such as BART and T5, improving their performance on the CNN/Dailymail and XSum benchmark datasets for abstractive summarization. This is achieved via fine-tuning, without requiring training from scratch. Finally, we show through human evaluation that our model generates more faithful summaries outperforming the state-of-the-art Frost model.