VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting
This addresses computational inefficiency for weather forecasting practitioners, but it is incremental as it builds on existing mixture-of-experts approaches.
The paper tackles the problem of high computational cost and continuous updating in weather forecasting by introducing VA-MoE, a framework that dynamically adapts to spatiotemporal patterns, achieving comparable accuracy to state-of-the-art models with only about 25% of trainable parameters and 50% of initial training data.
This paper presents Variables Adaptive Mixture of Experts (VAMoE), a novel framework for incremental weather forecasting that dynamically adapts to evolving spatiotemporal patterns in real time data. Traditional weather prediction models often struggle with exorbitant computational expenditure and the need to continuously update forecasts as new observations arrive. VAMoE addresses these challenges by leveraging a hybrid architecture of experts, where each expert specializes in capturing distinct subpatterns of atmospheric variables (temperature, humidity, wind speed). Moreover, the proposed method employs a variable adaptive gating mechanism to dynamically select and combine relevant experts based on the input context, enabling efficient knowledge distillation and parameter sharing. This design significantly reduces computational overhead while maintaining high forecast accuracy. Experiments on real world ERA5 dataset demonstrate that VAMoE performs comparable against SoTA models in both short term (1 days) and long term (5 days) forecasting tasks, with only about 25% of trainable parameters and 50% of the initial training data.