Valid Error Bars for Neural Weather Models using Conformal Prediction
This work addresses the lack of uncertainty estimates in neural weather models, which limits trust and usefulness for meteorologists and forecast users, though it is incremental as it applies an existing method to a new domain.
The authors tackled the problem of uncertainty quantification in neural weather models by developing a conformal prediction framework as a post-processing method, resulting in calibrated error bounds for all variables, lead times, and spatial locations with negligible computational cost.
Neural weather models have shown immense potential as inexpensive and accurate alternatives to physics-based models. However, most models trained to perform weather forecasting do not quantify the uncertainty associated with their forecasts. This limits the trust in the model and the usefulness of the forecasts. In this work we construct and formalise a conformal prediction framework as a post-processing method for estimating this uncertainty. The method is model-agnostic and gives calibrated error bounds for all variables, lead times and spatial locations. No modifications are required to the model and the computational cost is negligible compared to model training. We demonstrate the usefulness of the conformal prediction framework on a limited area neural weather model for the Nordic region. We further explore the advantages of the framework for deterministic and probabilistic models.