Uncertainty quantification via conformal prediction in data assimilation
For researchers in numerical weather prediction and data assimilation, this work demonstrates the potential of conformal prediction as a complementary uncertainty quantification tool, though in an idealized setting.
This study applies conformal prediction (CP) to quantify uncertainty in a simplified atmospheric model (1D modified shallow water). Results show that CP methods provide valid coverage and can complement traditional ensemble-based uncertainty quantification, with normalized CP and conformalized quantile regression offering improved interval scores.
Quantifying the evolution of uncertainty is critical to both probabilistic forecasting and data assimilation in numerical weather prediction. In this study, we investigate the applicability of conformal prediction (CP), a recent machine learning (ML) method, to quantify uncertainty in a controlled, idealized setting. We use the one dimensional modified shallow water model, designed to mimic the convective process. CP provides a set of possible outcomes with a chosen confidence level. Here, we compare and evaluate the average empirical coverage, the average interval length, miss low, miss high and average interval score loss (AISL) for three variants of CP, namely a) Standard CP, b) Normalized CP and c) Conformalized Quantile Regression. We further compare these CP-based uncertainty estimates with traditional ensemble-based measures such as standard deviation intervals and ensemble spread. In addition, we investigate the integration of CP-derived uncertainty within the data assimilation cycle through CP perturbations. Our results highlight the strengths and limitations of each approach, providing insight into the effectiveness of CP to complement common ensemble-based uncertainty quantification in simplified atmospheric models.