FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting
This work solves the challenge of accurate time series forecasting for domains like energy and weather by introducing a novel attention mechanism, though it is incremental as it builds on existing channel attention methods.
The paper tackles the problem of time series forecasting by addressing the inability of mainstream models to capture frequency information, which often leads to poor predictions; it proposes a frequency enhanced channel attention mechanism using Discrete Cosine Transform to avoid noise from Fourier Transform, achieving state-of-the-art performance with reductions in MSE of up to 35.99% on various models.
Time series forecasting is a long-standing challenge due to the real-world information is in various scenario (e.g., energy, weather, traffic, economics, earthquake warning). However some mainstream forecasting model forecasting result is derailed dramatically from ground truth. We believe it's the reason that model's lacking ability of capturing frequency information which richly contains in real world datasets. At present, the mainstream frequency information extraction methods are Fourier transform(FT) based. However, use of FT is problematic due to Gibbs phenomenon. If the values on both sides of sequences differ significantly, oscillatory approximations are observed around both sides and high frequency noise will be introduced. Therefore We propose a novel frequency enhanced channel attention that adaptively modelling frequency interdependencies between channels based on Discrete Cosine Transform which would intrinsically avoid high frequency noise caused by problematic periodity during Fourier Transform, which is defined as Gibbs Phenomenon. We show that this network generalize extremely effectively across six real-world datasets and achieve state-of-the-art performance, we further demonstrate that frequency enhanced channel attention mechanism module can be flexibly applied to different networks. This module can improve the prediction ability of existing mainstream networks, which reduces 35.99% MSE on LSTM, 10.01% on Reformer, 8.71% on Informer, 8.29% on Autoformer, 8.06% on Transformer, etc., at a slight computational cost ,with just a few line of code. Our codes and data are available at https://github.com/Zero-coder/FECAM.