Generating Summaries for Scientific Paper Review
This addresses the workload and potential quality issues for reviewers in scientific publishing, but it is incremental as it builds on existing summarization methods.
The paper tackles the problem of excessive reviewer burden in top ML/NLP venues by exploring automatic review summary generation, releasing a new dataset of NeurIPS papers and reviews from 2013-2020 and evaluating state-of-the-art neural summarization models to assess feasibility.
The review process is essential to ensure the quality of publications. Recently, the increase of submissions for top venues in machine learning and NLP has caused a problem of excessive burden on reviewers and has often caused concerns regarding how this may not only overload reviewers, but also may affect the quality of the reviews. An automatic system for assisting with the reviewing process could be a solution for ameliorating the problem. In this paper, we explore automatic review summary generation for scientific papers. We posit that neural language models have the potential to be valuable candidates for this task. In order to test this hypothesis, we release a new dataset of scientific papers and their reviews, collected from papers published in the NeurIPS conference from 2013 to 2020. We evaluate state of the art neural summarization models, present initial results on the feasibility of automatic review summary generation, and propose directions for the future.