AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement
This addresses the problem of timely and fair proposal review for astronomers and observatories, offering an incremental improvement through automation and iterative feedback.
The authors tackled the bottleneck of peer review for telescope proposals by developing AstroReview, an LLM-driven multi-agent framework that automates review stages, achieving 87% accuracy in identifying accepted proposals and increasing acceptance rates of revised drafts by 66% after two iterations.
Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astronomy. Automating parts of this process is therefore both scientifically significant and operationally necessary to ensure fair allocation and reproducible decisions at scale. We present AstroReview, an open-source, agent-based framework that automates proposal review in three stages: (i) novelty and scientific merit, (ii) feasibility and expected yield, and (iii) meta-review and reliability verification. Task isolation and explicit reasoning traces curb hallucinations and improve transparency. Without any domain specific fine tuning, AstroReview used in our experiments only for the last stage, correctly identifies genuinely accepted proposals with an accuracy of 87%. The AstroReview in Action module replicates the review and refinement loop; with its integrated Proposal Authoring Agent, the acceptance rate of revised drafts increases by 66% after two iterations, showing that iterative feedback combined with automated meta-review and reliability verification delivers measurable quality gains. Together, these results point to a practical path toward scalable, auditable, and higher throughput proposal review for resource limited facilities.