How Foundational are Foundation Models for Time Series Forecasting?
This challenges the assumption that foundation models are universally effective for time series, which is important for researchers and practitioners in time series analysis and machine learning, indicating an incremental insight.
The paper tackles the problem of applying foundation models to time series forecasting, showing that their zero-shot capabilities are domain-dependent and fine-tuning does not consistently outperform smaller task-specific models in terms of results relative to computational cost.
Foundation Models are designed to serve as versatile embedding machines, with strong zero shot capabilities and superior generalization performance when fine-tuned on diverse downstream tasks. While this is largely true for language and vision foundation models, we argue that the inherent diversity of time series data makes them less suited for building effective foundation models. We demonstrate this using forecasting as our downstream task. We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on. Furthermore, when applied to unseen real-world time series data, fine-tuned foundation models do not consistently yield substantially better results, relative to their increased parameter count and memory footprint, than smaller, dedicated models tailored to the specific forecasting task at hand.