Judgment-Grounded Expansion for Peer Review Generation
This work establishes a new task and methodology for human-AI collaborative review generation, addressing the need for accountability in automated review systems.
The paper introduces judgment-grounded expansion, a human-AI collaboration mode for peer review generation where a reviewer provides an evaluative claim and the system expands it into review comments. The authors model this as a structured generate-check-refine process and demonstrate that conformal prediction effectively balances candidate set size and target coverage.
Automatic review generation is a promising direction for accelerating scientific progress. While most work adopts an end-to-end setup, its fully automated nature makes it less suitable for settings that demand accountability. To better balance automation and accountability, we formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s). We model it as a structured generate-check-refine process and conduct a user study to collect human-model interaction data. We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation. We develop methods to simulate the process for large-scale evaluation, and show that conformal prediction is well suited to balancing candidate set size and target coverage. Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.