Position on LLM-Assisted Peer Review: Addressing Reviewer Gap through Mentoring and Feedback
It tackles the problem of low-quality peer reviews for the AI research community, offering a human-centered approach rather than an incremental technical improvement.
This position paper addresses the Reviewer Gap in AI research by proposing a paradigm shift where LLMs assist and educate human reviewers, aiming to strengthen reviewer expertise and build a more sustainable scholarly ecosystem.
The rapid expansion of AI research has intensified the Reviewer Gap, threatening the peer-review sustainability and perpetuating a cycle of low-quality evaluations. This position paper critiques existing LLM approaches that automatically generate reviews and argues for a paradigm shift that positions LLMs as tools for assisting and educating human reviewers. We define the core principles of high-quality peer review and propose two complementary systems grounded in these foundations: (i) an LLM-assisted mentoring system that cultivates reviewers' long-term competencies, and (ii) an LLM-assisted feedback system that helps reviewers refine the quality of their reviews. This human-centered approach aims to strengthen reviewer expertise and contribute to building a more sustainable scholarly ecosystem.