Foundation Models for Time Series: A Survey
It addresses the need for a structured overview in time series analysis, which is incremental as it synthesizes existing work without proposing new methods.
This survey tackles the problem of organizing the rapidly evolving field of transformer-based foundation models for time series analysis by introducing a novel taxonomy that categorizes models across multiple dimensions, such as architecture, prediction type, and training objectives, to serve as a comprehensive resource for researchers and practitioners.
Transformer-based foundation models have emerged as a dominant paradigm in time series analysis, offering unprecedented capabilities in tasks such as forecasting, anomaly detection, classification, trend analysis and many more time series analytical tasks. This survey provides a comprehensive overview of the current state of the art pre-trained foundation models, introducing a novel taxonomy to categorize them across several dimensions. Specifically, we classify models by their architecture design, distinguishing between those leveraging patch-based representations and those operating directly on raw sequences. The taxonomy further includes whether the models provide probabilistic or deterministic predictions, and whether they are designed to work with univariate time series or can handle multivariate time series out of the box. Additionally, the taxonomy encompasses model scale and complexity, highlighting differences between lightweight architectures and large-scale foundation models. A unique aspect of this survey is its categorization by the type of objective function employed during training phase. By synthesizing these perspectives, this survey serves as a resource for researchers and practitioners, providing insights into current trends and identifying promising directions for future research in transformer-based time series modeling.