Decision-oriented benchmarking to transform AI weather forecast access: Application to the Indian monsoon
This work addresses the challenge of making AI weather forecasts useful for vulnerable populations like Indian farmers facing climate change, though it is incremental in applying existing AI models within a new benchmarking framework.
The paper tackles the problem that AI weather prediction models are not evaluated based on local stakeholders' decision needs, introducing a framework that connects meteorology, AI, and social sciences. As a result, it applied this to Indian monsoon forecasting, enabling AI models to skillfully predict agriculturally relevant onset weeks in advance and informing a 2025 government effort that sent forecasts to 38 million farmers, capturing an unusual monsoon pause.
Artificial intelligence weather prediction (AIWP) models now often outperform traditional physics-based models on common metrics while requiring orders-of-magnitude less computing resources and time. Open-access AIWP models thus hold promise as transformational tools for helping low- and middle-income populations make decisions in the face of high-impact weather shocks. Yet, current approaches to evaluating AIWP models focus mainly on aggregated meteorological metrics without considering local stakeholders' needs in decision-oriented, operational frameworks. Here, we introduce such a framework that connects meteorology, AI, and social sciences. As an example, we apply it to the 150-year-old problem of Indian monsoon forecasting, focusing on benefits to rain-fed agriculture, which is highly susceptible to climate change. AIWP models skillfully predict an agriculturally relevant onset index at regional scales weeks in advance when evaluated out-of-sample using deterministic and probabilistic metrics. This framework informed a government-led effort in 2025 to send 38 million Indian farmers AI-based monsoon onset forecasts, which captured an unusual weeks-long pause in monsoon progression. This decision-oriented benchmarking framework provides a key component of a blueprint for harnessing the power of AIWP models to help large vulnerable populations adapt to weather shocks in the face of climate variability and change.