WeatherFormer: Empowering Global Numerical Weather Forecasting with Space-Time Transformer
This work addresses the need for efficient and eco-friendly weather forecasting solutions, which is a critical problem for the AI and earth science communities, though it represents an incremental improvement by building on transformer architectures.
The paper tackles the inefficiency and high carbon emissions of traditional Numerical Weather Prediction systems by proposing WeatherFormer, a transformer-based framework that models spatio-temporal dynamics and reduces parameters and memory usage, achieving superior performance over existing deep learning methods and approaching state-of-the-art physical models on the WeatherBench dataset.
Numerical Weather Prediction (NWP) system is an infrastructure that exerts considerable impacts on modern society.Traditional NWP system, however, resolves it by solving complex partial differential equations with a huge computing cluster, resulting in tons of carbon emission. Exploring efficient and eco-friendly solutions for NWP attracts interest from Artificial Intelligence (AI) and earth science communities. To narrow the performance gap between the AI-based methods and physic predictor, this work proposes a new transformer-based NWP framework, termed as WeatherFormer, to model the complex spatio-temporal atmosphere dynamics and empowering the capability of data-driven NWP. WeatherFormer innovatively introduces the space-time factorized transformer blocks to decrease the parameters and memory consumption, in which Position-aware Adaptive Fourier Neural Operator (PAFNO) is proposed for location sensible token mixing. Besides, two data augmentation strategies are utilized to boost the performance and decrease training consumption. Extensive experiments on WeatherBench dataset show WeatherFormer achieves superior performance over existing deep learning methods and further approaches the most advanced physical model.