Exploring Design Choices for Autoregressive Deep Learning Climate Models
This work addresses the challenge of long-term stability in deep learning climate models for weather prediction, though it is incremental as it builds on existing architectures.
This study tackled the problem of deep learning climate models failing to maintain physically consistent rollouts beyond 14 days by comparing three architectures (FourCastNet, SFNO, ClimaX) to identify design choices enabling stable 10-year rollouts while preserving statistical properties of reference data.
Deep Learning models have achieved state-of-the-art performance in medium-range weather prediction but often fail to maintain physically consistent rollouts beyond 14 days. In contrast, a few atmospheric models demonstrate stability over decades, though the key design choices enabling this remain unclear. This study quantitatively compares the long-term stability of three prominent DL-MWP architectures - FourCastNet, SFNO, and ClimaX - trained on ERA5 reanalysis data at 5.625° resolution. We systematically assess the impact of autoregressive training steps, model capacity, and choice of prognostic variables, identifying configurations that enable stable 10-year rollouts while preserving the statistical properties of the reference dataset. Notably, rollouts with SFNO exhibit the greatest robustness to hyperparameter choices, yet all models can experience instability depending on the random seed and the set of prognostic variables