MultiResFormer: Transformer with Adaptive Multi-Resolution Modeling for General Time Series Forecasting
This addresses the challenge of modeling multi-periodic time series for forecasting applications, representing an incremental improvement over existing transformer methods.
The paper tackles the problem of capturing diverse temporal dependencies in time series forecasting by proposing MultiResFormer, a transformer model that adaptively selects optimal patch lengths, resulting in outperforming patch-based transformers on long-term tasks and CNN baselines by a large margin with fewer parameters.
Transformer-based models have greatly pushed the boundaries of time series forecasting recently. Existing methods typically encode time series data into $\textit{patches}$ using one or a fixed set of patch lengths. This, however, could result in a lack of ability to capture the variety of intricate temporal dependencies present in real-world multi-periodic time series. In this paper, we propose MultiResFormer, which dynamically models temporal variations by adaptively choosing optimal patch lengths. Concretely, at the beginning of each layer, time series data is encoded into several parallel branches, each using a detected periodicity, before going through the transformer encoder block. We conduct extensive evaluations on long- and short-term forecasting datasets comparing MultiResFormer with state-of-the-art baselines. MultiResFormer outperforms patch-based Transformer baselines on long-term forecasting tasks and also consistently outperforms CNN baselines by a large margin, while using much fewer parameters than these baselines.