FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks
This addresses a universal issue in time-series prediction models, offering a plug-and-play solution for improved generalization, though it is incremental as it builds on existing observations of spectral bias.
The paper tackles the challenge of spectral bias in neural networks for time-series prediction, where models favor low-frequency signals, and proposes the FreLE algorithm to mitigate this through frequency regularization, achieving superior performance in experiments.
The inherent autocorrelation of time series data presents an ongoing challenge to multivariate time series prediction. Recently, a widely adopted approach has been the incorporation of frequency domain information to assist in long-term prediction tasks. Many researchers have independently observed the spectral bias phenomenon in neural networks, where models tend to fit low-frequency signals before high-frequency ones. However, these observations have often been attributed to the specific architectures designed by the researchers, rather than recognizing the phenomenon as a universal characteristic across models. To unify the understanding of the spectral bias phenomenon in long-term time series prediction, we conducted extensive empirical experiments to measure spectral bias in existing mainstream models. Our findings reveal that virtually all models exhibit this phenomenon. To mitigate the impact of spectral bias, we propose the FreLE (Frequency Loss Enhancement) algorithm, which enhances model generalization through both explicit and implicit frequency regularization. This is a plug-and-play model loss function unit. A large number of experiments have proven the superior performance of FreLE. Code is available at https://github.com/Chenxing-Xuan/FreLE.