In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks
This addresses the need for more adaptable time-series models in domains like forecasting or anomaly detection, though it is incremental as it builds on existing foundation models.
The paper tackled the problem of time-series foundation models struggling to generalize to unseen tasks without fine-tuning, and the result was that their proposed In-Context Time-series Pre-training framework improved performance by approximately 11.4% on unseen tasks.
Time-series foundation models (TSFMs) have demonstrated strong generalization capabilities across diverse datasets and tasks. However, existing foundation models are typically pre-trained to enhance performance on specific tasks and often struggle to generalize to unseen tasks without fine-tuning. To address this limitation, we propose augmenting TSFMs with In-Context Learning (ICL) capabilities, enabling them to perform test-time inference by dynamically adapting to input-output relationships provided within the context. Our framework, In-Context Time-series Pre-training (ICTP), restructures the original pre-training data to equip the backbone TSFM with ICL capabilities, enabling adaptation to unseen tasks. Experiments demonstrate that ICT improves the performance of state-of-the-art TSFMs by approximately 11.4% on unseen tasks without requiring fine-tuning.