Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification
This challenges the assumption that task-specific pre-training is necessary for time series classification, potentially simplifying model development for practitioners in fields like finance or healthcare.
The study investigated whether frozen pre-trained forecasting models can serve as effective feature extractors for time series classification, finding that they achieve classification accuracy matching or surpassing state-of-the-art models pre-trained specifically for classification, with a positive correlation between forecasting and classification performance.
Recent research on time series foundation models has primarily focused on forecasting, leaving it unclear how generalizable their learned representations are. In this study, we examine whether frozen pre-trained forecasting models can provide effective representations for classification. To this end, we compare different representation extraction strategies and introduce two model-agnostic embedding augmentations. Our experiments show that the best forecasting models achieve classification accuracy that matches or even surpasses that of state-of-the-art models pre-trained specifically for classification. Moreover, we observe a positive correlation between forecasting and classification performance. These findings challenge the assumption that task-specific pre-training is necessary, and suggest that learning to forecast may provide a powerful route toward constructing general-purpose time series foundation models.