Towards grounded autonomous research: an end-to-end LLM mini research loop on published computational physics
This work demonstrates the first autonomous agent capable of performing the full research loop (read, reproduce, critique, extend) on published computational physics papers, with concrete evidence of generating a publishable revision of a headline conclusion.
The authors developed an end-to-end LLM agent that autonomously reads, reproduces, critiques, and extends computational physics papers. In a scale test across 111 papers, it raised substantive concerns on ~42% of papers, with 97.7% requiring execution to surface; in a depth test, it produced an unsupervised publishable Comment revising a Nature Communications paper's headline conclusion.
Recent autonomous LLM agents have demonstrated end-to-end automation of machine-learning research. Real-world physical science is intrinsically harder, requiring deep reasoning bounded by physical truth and, because real systems are too complex to study in isolation, almost always built on existing literature. We focus on the smallest meaningful unit of such research, a mini research loop in which an agent reads a paper, reproduces it, critiques it, and extends it. We test this loop in two complementary regimes: scale and depth. At scale, across 111 open-access computational physics papers, an agent autonomously runs the read-plan-compute-compare loop and, without being asked to critique, raises substantive concerns on ~42% of papers - 97.7% of which require execution to surface. In depth, for one Nature Communications paper on multiscale simulation of a 2D-material MOSFET, the agent runs new calculations missing from the original and produces, unsupervised, a publishable Comment -- composed, figured, typeset, and PDF-iterated -- that revises the paper's headline conclusion.