GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews
This addresses the burden on area chairs in peer review by providing a pragmatic overview of reviews, though it is an incremental improvement over existing summarization methods.
The paper tackles the problem of summarizing scholarly reviews for area chairs by introducing GLIMPSE, a method that extracts both common and unique opinions using uniqueness scores based on the Rational Speech Act framework, resulting in more discriminative summaries in human evaluation while matching baseline performance in automatic metrics.
Scientific peer review is essential for the quality of academic publications. However, the increasing number of paper submissions to conferences has strained the reviewing process. This surge poses a burden on area chairs who have to carefully read an ever-growing volume of reviews and discern each reviewer's main arguments as part of their decision process. In this paper, we introduce \sys, a summarization method designed to offer a concise yet comprehensive overview of scholarly reviews. Unlike traditional consensus-based methods, \sys extracts both common and unique opinions from the reviews. We introduce novel uniqueness scores based on the Rational Speech Act framework to identify relevant sentences in the reviews. Our method aims to provide a pragmatic glimpse into all reviews, offering a balanced perspective on their opinions. Our experimental results with both automatic metrics and human evaluation show that \sys generates more discriminative summaries than baseline methods in terms of human evaluation while achieving comparable performance with these methods in terms of automatic metrics.