Self-Attentive Ensemble Transformer: Representing Ensemble Interactions in Neural Networks for Earth System Models
This work addresses the need for non-parametric post-processing of ensemble data in Earth system modeling, offering a novel method for improving forecast calibration and spatial coherence, though it is incremental as it builds on existing transformer and ensemble techniques.
The paper tackles the problem of calibrating and post-processing ensemble data from Earth system models by proposing a self-attentive ensemble transformer, which uses self-attention to represent interactions between ensemble members, and demonstrates its ability to calibrate ensemble spread and extract additional information from global ECMWF ensemble forecasts for 2-metre-temperature fields.
Ensemble data from Earth system models has to be calibrated and post-processed. I propose a novel member-by-member post-processing approach with neural networks. I bridge ideas from ensemble data assimilation with self-attention, resulting into the self-attentive ensemble transformer. Here, interactions between ensemble members are represented as additive and dynamic self-attentive part. As proof-of-concept, I regress global ECMWF ensemble forecasts to 2-metre-temperature fields from the ERA5 reanalysis. I demonstrate that the ensemble transformer can calibrate the ensemble spread and extract additional information from the ensemble. As it is a member-by-member approach, the ensemble transformer directly outputs multivariate and spatially-coherent ensemble members. Therefore, self-attention and the transformer technique can be a missing piece for a non-parametric post-processing of ensemble data with neural networks.