EMAformer: Enhancing Transformer through Embedding Armor for Time Series Forecasting
This work addresses the problem of improving Transformer-based models for multivariate time series forecasting, which is crucial across various domains, by offering a novel enhancement that advances practical applicability.
The paper tackles the performance gap of Transformer models in multivariate time series forecasting by proposing EMAformer, which introduces an auxiliary embedding suite with inductive biases to stabilize inter-channel relationships, achieving state-of-the-art results on 12 benchmarks with average error reductions of 2.73% in MSE and 5.15% in MAE.
Multivariate time series forecasting is crucial across a wide range of domains. While presenting notable progress for the Transformer architecture, iTransformer still lags behind the latest MLP-based models. We attribute this performance gap to unstable inter-channel relationships. To bridge this gap, we propose EMAformer, a simple yet effective model that enhances the Transformer with an auxiliary embedding suite, akin to armor that reinforces its ability. By introducing three key inductive biases, i.e., \textit{global stability}, \textit{phase sensitivity}, and \textit{cross-axis specificity}, EMAformer unlocks the further potential of the Transformer architecture, achieving state-of-the-art performance on 12 real-world benchmarks and reducing forecasting errors by an average of 2.73\% in MSE and 5.15\% in MAE. This significantly advances the practical applicability of Transformer-based approaches for multivariate time series forecasting. The code is available on https://github.com/PlanckChang/EMAformer.