A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization
This addresses the challenge of generating coherent summaries from multiple documents, which is incremental as it builds on pre-trained language models by adding hierarchy to improve cross-document attention.
The paper tackled the problem of abstractive multi-document summarization (MDS) by proposing a hierarchical encoding-decoding scheme to better handle cross-document interactions, resulting in performance that outperforms or is competitive with previous best models across 10 benchmarks, with up to a 3 Rouge-L improvement over the backbone PLM and human preference.
Pre-trained language models (PLMs) have achieved outstanding achievements in abstractive single-document summarization (SDS). However, such benefits may not fully extend to multi-document summarization (MDS), where the handling of cross-document information is more complex. Previous works either design new MDS architectures or apply PLMs bluntly with concatenated source documents as a reformulated SDS task. While the former does not utilize previous pre-training efforts and may not generalize well across different domains, the latter may not sufficiently attend to the intricate cross-document relationships unique to MDS tasks. Instead, we enforce hierarchy on both the encoder and decoder to better utilize a PLM to facilitate multi-document interactions for the MDS task. Across 10 MDS benchmarks from various domains, our method outperforms or is competitive with the previous best models, including those with additional MDS pre-training or with more parameters. It outperforms its corresponding PLM backbone by up to 3 Rouge-L and is favored by humans.