CMIP-Forge: An Agentic System that Retrieves, Computes, and Self-Reviews Climate Science
For climate scientists transitioning to CMIP7, CMIP-Forge automates complex literature-driven data analysis, but its novelty is incremental as it combines existing RAG and agentic techniques with domain-specific guardrails.
CMIP-Forge is a hybrid RAG and autonomous analysis system that retrieves and operationalizes knowledge from 6,581 CMIP6 publications alongside live ESGF data, enabling end-to-end autonomous climate research pipelines. The system demonstrates capabilities across atmospheric teleconnections, ocean dynamics, and global warming projections, while exposing concrete failure modes in its adversarial review loop.
The Coupled Model Intercomparison Project Phase 6 (CMIP6) has generated thousands of peer-reviewed publications documenting model configurations, evaluation procedures, emergent constraints, and projection uncertainties. As the community transitions toward CMIP7, efficiently extracting and operationalizing this unstructured knowledge alongside live data analysis represents a critical bottleneck. Here we present CMIP-Forge, a hybrid retrieval-augmented generation (RAG) and autonomous analysis system that bridges the gap between scientific literature and Earth System Grid Federation (ESGF) data archives. The system pairs a curated corpus of 6,581 CMIP6-related open-access publications (101,828 indexed chunks) with an agentic pipeline in which a tool-augmented worker plans and executes Python workflows over live climate data, while a panel of independent reviewer models audits its methodology end to end. CMIP-Forge introduces a multi-layered Defense-in-Depth architecture that enforces physical and methodological invariants through executable mechanisms: Abstract Syntax Tree (AST) static analysis, audited scientific primitives, and an autonomous adversarial peer-review protocol. We demonstrate the system's capabilities through end-to-end autonomous research pipelines spanning atmospheric teleconnections, ocean dynamics, regional extremes, and global warming projections. An agentic analysis system grounded in peer-reviewed literature, constrained by automated code guardrails, and audited by an independent adversarial review loop can complete complex climate-research workflows autonomously. The same experiments expose concrete failure modes of the review loop (sycophantic regression, REVISE verdicts that are never resolved, and the submission of stub code for review), each diagnosable from the immutable telemetry and provenance record released with the article.