No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation
For climate scientists and ML practitioners building climate emulators, this work identifies a critical blind spot and provides a principled evaluation framework and solution direction.
This paper shows that climate emulation is an out-of-distribution (OOD) projection task where current ML emulators fail under distribution shifts. It introduces a novel evaluation framework using seasonal shifts as a proxy for long-term climate shifts, demonstrates that state-of-the-art hybrid-ML emulators degrade significantly, and proposes compositional generalization as a path to robust emulation.
Climate emulation is an out-of-distribution (OOD) projection task. This is precisely the challenge where modern Machine Learning (ML) methods are most prone to failure. Consequently, while current ML emulators trained on present climate achieve high in-distribution performance, their future reliability under the inevitable distribution shifts of a changing climate remains a critical, poorly understood blind spot. Addressing this challenge requires a fundamental shift in how we understand, evaluate, and design climate emulators. In this work, we first confirm that climate change drives a statistically significant and progressively growing shift in atmospheric state distributions, rendering standard evaluation protocols insufficient. We empirically establish that seasonal variation serves as an effective proxy for these long-term climate shifts, providing access to $\textit{real-world}$ distribution shifts without recourse to heuristics like synthetic perturbations. Motivated by this link, we introduce a novel evaluation framework that leverages seasonal shifts as a rigorous, zero-overhead testbed for emulator robustness. Our systematic characterisation confirms that current state-of-the-art hybrid-ML emulators degrade significantly under these realistic shifts. Finally, we chart a path forward by identifying compositional generalisation, the ability to form novel combinations from observed elementary components, as a principled route towards robust climate emulation. We demonstrate that physically motivated decompositions substantially improve OOD performance with only modest trade-offs against in-distribution performance, providing an avenue towards ML-driven climate emulators robust to an unknown future.