ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning
This work solves the challenge of accurate multivariate forecasting for domains relying on time series data, representing an incremental improvement over existing methods.
The paper tackles the problem of multivariate long-term time series forecasting by addressing inefficiencies in existing Transformers that incorrectly capture series-wise relationships, resulting in ARM, which demonstrates superior performance on multiple benchmarks without significantly increasing computational costs.
Long-term time series forecasting (LTSF) is important for various domains but is confronted by challenges in handling the complex temporal-contextual relationships. As multivariate input models underperforming some recent univariate counterparts, we posit that the issue lies in the inefficiency of existing multivariate LTSF Transformers to model series-wise relationships: the characteristic differences between series are often captured incorrectly. To address this, we introduce ARM: a multivariate temporal-contextual adaptive learning method, which is an enhanced architecture specifically designed for multivariate LTSF modelling. ARM employs Adaptive Univariate Effect Learning (AUEL), Random Dropping (RD) training strategy, and Multi-kernel Local Smoothing (MKLS), to better handle individual series temporal patterns and correctly learn inter-series dependencies. ARM demonstrates superior performance on multiple benchmarks without significantly increasing computational costs compared to vanilla Transformer, thereby advancing the state-of-the-art in LTSF. ARM is also generally applicable to other LTSF architecture beyond vanilla Transformer.