PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
This addresses the problem of automating scientific writing for AI researchers, though it appears incremental as it builds on existing autonomous writing methods.
The paper tackles the challenge of synthesizing unstructured research materials into manuscripts by introducing PaperOrchestra, a multi-agent framework for automated AI research paper writing, which outperforms baselines with win rate margins of 50%-68% in literature review quality and 14%-38% in overall manuscript quality.
Synthesizing unstructured research materials into manuscripts is an essential yet under-explored challenge in AI-driven scientific discovery. Existing autonomous writers are rigidly coupled to specific experimental pipelines, and produce superficial literature reviews. We introduce PaperOrchestra, a multi-agent framework for automated AI research paper writing. It flexibly transforms unconstrained pre-writing materials into submission-ready LaTeX manuscripts, including comprehensive literature synthesis and generated visuals, such as plots and conceptual diagrams. To evaluate performance, we present PaperWritingBench, the first standardized benchmark of reverse-engineered raw materials from 200 top-tier AI conference papers, alongside a comprehensive suite of automated evaluators. In side-by-side human evaluations, PaperOrchestra significantly outperforms autonomous baselines, achieving an absolute win rate margin of 50%-68% in literature review quality, and 14%-38% in overall manuscript quality.