A Sentiment Consolidation Framework for Meta-Review Generation
This addresses the challenge of sentiment summarization in scientific domains, but it is incremental as it builds on existing LLM capabilities.
The paper tackles the problem of generating meta-reviews by proposing a three-layer sentiment consolidation framework, and finds that prompting LLMs with this framework yields better results compared to simple instructions.
Modern natural language generation systems with Large Language Models (LLMs) exhibit the capability to generate a plausible summary of multiple documents; however, it is uncertain if they truly possess the capability of information consolidation to generate summaries, especially on documents with opinionated information. We focus on meta-review generation, a form of sentiment summarisation for the scientific domain. To make scientific sentiment summarization more grounded, we hypothesize that human meta-reviewers follow a three-layer framework of sentiment consolidation to write meta-reviews. Based on the framework, we propose novel prompting methods for LLMs to generate meta-reviews and evaluation metrics to assess the quality of generated meta-reviews. Our framework is validated empirically as we find that prompting LLMs based on the framework -- compared with prompting them with simple instructions -- generates better meta-reviews.