Detecting AI-Generated Content in Academic Peer Reviews
This addresses the issue of AI influence in scholarly evaluation for academic communities, but it is incremental as it applies an existing detection method to new data.
This study tackled the problem of detecting AI-generated content in academic peer reviews by analyzing temporal trends, finding that AI-generated reviews increased substantially from minimal levels before 2022 to approximately 20% in ICLR and 12% in Nature Communications by 2025.
The growing availability of large language models (LLMs) has raised questions about their role in academic peer review. This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews to later review cycles at International Conference on Learning Representations (ICLR) and Nature Communications (NC). We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025, with approximately 20% of ICLR reviews and 12% of Nature Communications reviews classified as AI-generated in 2025. The most pronounced growth of AI-generated reviews in NC occurs between the third and fourth quarter of 2024. Together, these findings provide suggestive evidence of a rapidly increasing presence of AI-assisted content in peer review and highlight the need for further study of its implications for scholarly evaluation.