Analyzing Sentence Fusion in Abstractive Summarization
This addresses the problem of understanding and improving faithfulness in abstractive summarization for NLP researchers, but it is incremental as it focuses on analysis rather than proposing a new method.
The paper analyzed how five state-of-the-art abstractive summarizers combine information from multiple document sentences through sentence fusion, finding that while system sentences are mostly grammatical, they often fail to remain faithful to the original article.
While recent work in abstractive summarization has resulted in higher scores in automatic metrics, there is little understanding on how these systems combine information taken from multiple document sentences. In this paper, we analyze the outputs of five state-of-the-art abstractive summarizers, focusing on summary sentences that are formed by sentence fusion. We ask assessors to judge the grammaticality, faithfulness, and method of fusion for summary sentences. Our analysis reveals that system sentences are mostly grammatical, but often fail to remain faithful to the original article.