The Power of Summary-Source Alignments
This work addresses the problem of improving multi-document summarization for researchers by providing new datasets and baselines, though it is incremental as it builds on existing alignment frameworks.
The paper tackles the challenge of multi-document summarization by extending summary-source alignments to a fine-grained proposition span level and manually annotating them, resulting in the release of datasets for at least six different tasks, including test sets with manual annotations and baseline models.
Multi-document summarization (MDS) is a challenging task, often decomposed to subtasks of salience and redundancy detection, followed by text generation. In this context, alignment of corresponding sentences between a reference summary and its source documents has been leveraged to generate training data for some of the component tasks. Yet, this enabling alignment step has usually been applied heuristically on the sentence level on a limited number of subtasks. In this paper, we propose extending the summary-source alignment framework by (1) applying it at the more fine-grained proposition span level, (2) annotating alignment manually in a multi-document setup, and (3) revealing the great potential of summary-source alignments to yield several datasets for at least six different tasks. Specifically, for each of the tasks, we release a manually annotated test set that was derived automatically from the alignment annotation. We also release development and train sets in the same way, but from automatically derived alignments. Using the datasets, each task is demonstrated with baseline models and corresponding evaluation metrics to spur future research on this broad challenge.