PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting
This work addresses a fundamental challenge in long-term time series forecasting for applications requiring accurate predictions, though it is incremental as it builds on existing CNN approaches.
The authors tackled the loss of temporal information in Transformer-based time series forecasting by proposing PatchMixer, a CNN-based model that uses depthwise separable convolutions and dual forecasting heads, achieving 3.9% and 21.2% relative improvements over state-of-the-art and best-performing CNN methods, respectively, with 2-3x faster inference.
Although the Transformer has been the dominant architecture for time series forecasting tasks in recent years, a fundamental challenge remains: the permutation-invariant self-attention mechanism within Transformers leads to a loss of temporal information. To tackle these challenges, we propose PatchMixer, a novel CNN-based model. It introduces a permutation-variant convolutional structure to preserve temporal information. Diverging from conventional CNNs in this field, which often employ multiple scales or numerous branches, our method relies exclusively on depthwise separable convolutions. This allows us to extract both local features and global correlations using a single-scale architecture. Furthermore, we employ dual forecasting heads encompassing linear and nonlinear components to better model future curve trends and details. Our experimental results on seven time-series forecasting benchmarks indicate that compared with the state-of-the-art method and the best-performing CNN, PatchMixer yields $3.9\%$ and $21.2\%$ relative improvements, respectively, while being 2-3x faster than the most advanced method.