PeerSum: A Peer Review Dataset for Abstractive Multi-document Summarization
This dataset addresses the need for more realistic and challenging summarization benchmarks in the NLP community, though it is incremental as it builds on existing MDS datasets.
The authors introduced PeerSum, a new multi-document summarization dataset based on peer reviews of scientific publications, which features highly abstractive summaries and source document disagreements. They found that current state-of-the-art models struggle with this dataset, highlighting new research challenges.
We present PeerSum, a new MDS dataset using peer reviews of scientific publications. Our dataset differs from the existing MDS datasets in that our summaries (i.e., the meta-reviews) are highly abstractive and they are real summaries of the source documents (i.e., the reviews) and it also features disagreements among source documents. We found that current state-of-the-art MDS models struggle to generate high-quality summaries for PeerSum, offering new research opportunities.