Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
This work addresses the challenge of generating high-quality summaries for long documents, which is crucial for applications like news aggregation and document analysis, by improving discourse-aware modeling.
The paper tackled the problem of long document abstractive summarization by incorporating both types and uncertainty of rhetorical relations from discourse structure, resulting in a model that significantly outperformed state-of-the-art models on automatic metrics and human evaluation.
For text summarization, the role of discourse structure is pivotal in discerning the core content of a text. Regrettably, prior studies on incorporating Rhetorical Structure Theory (RST) into transformer-based summarization models only consider the nuclearity annotation, thereby overlooking the variety of discourse relation types. This paper introduces the 'RSTformer', a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations. Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework. Through rigorous evaluation, the model proposed herein exhibits significant superiority over state-of-the-art models, as evidenced by its notable performance on several automatic metrics and human evaluation.