MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing
This work addresses forecasting challenges in practical scenarios like finance or weather, but it is incremental as it builds on existing token aggregation ideas from vision and NLP.
The paper tackles the problem of multivariate time series forecasting by investigating the role of attention mechanisms and finding they are unnecessary for capturing temporal dependencies, leading to the proposal of MTS-Mixers, which outperforms Transformer-based models with higher efficiency on real-world datasets.
Multivariate time series forecasting has been widely used in various practical scenarios. Recently, Transformer-based models have shown significant potential in forecasting tasks due to the capture of long-range dependencies. However, recent studies in the vision and NLP fields show that the role of attention modules is not clear, which can be replaced by other token aggregation operations. This paper investigates the contributions and deficiencies of attention mechanisms on the performance of time series forecasting. Specifically, we find that (1) attention is not necessary for capturing temporal dependencies, (2) the entanglement and redundancy in the capture of temporal and channel interaction affect the forecasting performance, and (3) it is important to model the mapping between the input and the prediction sequence. To this end, we propose MTS-Mixers, which use two factorized modules to capture temporal and channel dependencies. Experimental results on several real-world datasets show that MTS-Mixers outperform existing Transformer-based models with higher efficiency.