Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting
This work addresses a key limitation in time series forecasting for applications requiring multi-scale analysis, though it appears incremental as it builds on existing Transformer architectures.
The paper tackles the challenge of capturing multi-scale characteristics in time series forecasting by proposing Pathformer, a Transformer model with adaptive pathways that integrates temporal resolution and distance, achieving state-of-the-art performance on eleven real-world datasets.
Transformers for time series forecasting mainly model time series from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. We propose Pathformer, a multi-scale Transformer with adaptive pathways. It integrates both temporal resolution and temporal distance for multi-scale modeling. Multi-scale division divides the time series into different temporal resolutions using patches of various sizes. Based on the division of each scale, dual attention is performed over these patches to capture global correlations and local details as temporal dependencies. We further enrich the multi-scale Transformer with adaptive pathways, which adaptively adjust the multi-scale modeling process based on the varying temporal dynamics of the input, improving the accuracy and generalization of Pathformer. Extensive experiments on eleven real-world datasets demonstrate that Pathformer not only achieves state-of-the-art performance by surpassing all current models but also exhibits stronger generalization abilities under various transfer scenarios. The code is made available at https://github.com/decisionintelligence/pathformer.