Do machine learning climate models work in changing climate dynamics?
This addresses the reliability of ML for climate risk forecasting, which is critical for adaptation, but it is incremental as it applies existing OOD evaluation methodologies to climate data.
This research tackled the problem of machine learning climate models generalizing under distribution shifts, such as out-of-distribution events, by systematically evaluating state-of-the-art models in diverse scenarios, revealing notable performance variability.
Climate change is accelerating the frequency and severity of unprecedented events, deviating from established patterns. Predicting these out-of-distribution (OOD) events is critical for assessing risks and guiding climate adaptation. While machine learning (ML) models have shown promise in providing precise, high-speed climate predictions, their ability to generalize under distribution shifts remains a significant limitation that has been underexplored in climate contexts. This research systematically evaluates state-of-the-art ML-based climate models in diverse OOD scenarios by adapting established OOD evaluation methodologies to climate data. Experiments on large-scale datasets reveal notable performance variability across scenarios, shedding light on the strengths and limitations of current models. These findings underscore the importance of robust evaluation frameworks and provide actionable insights to guide the reliable application of ML for climate risk forecasting.