Self-Supervised Learning for Time Series: A Review & Critique of FITS
This work addresses the challenge of efficient and accurate time series forecasting for industries relying on predictive analytics, though it is incremental as it builds on existing models.
The paper tackles the problem of accurate time series forecasting by critiquing and improving upon the FITS model, which claims competitive performance with reduced parameters but struggles with non-periodic patterns; the authors propose two novel hybrid approaches combining FITS with DLinear, achieving state-of-the-art results on multivariate regression and significant improvements over standalone FITS.
Accurate time series forecasting is a highly valuable endeavour with applications across many industries. Despite recent deep learning advancements, increased model complexity, and larger model sizes, many state-of-the-art models often perform worse or on par with simpler models. One of those cases is a recently proposed model, FITS, claiming competitive performance with significantly reduced parameter counts. By training a one-layer neural network in the complex frequency domain, we are able to replicate these results. Our experiments on a wide range of real-world datasets further reveal that FITS especially excels at capturing periodic and seasonal patterns, but struggles with trending, non-periodic, or random-resembling behavior. With our two novel hybrid approaches, where we attempt to remedy the weaknesses of FITS by combining it with DLinear, we achieve the best results of any known open-source model on multivariate regression and promising results in multiple/linear regression on price datasets, on top of vastly improving upon what FITS achieves as a standalone model.