A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks
This addresses the need for flexible time-series analysis tools in domains like finance or healthcare, though it builds incrementally on existing in-context learning and foundation model concepts.
The paper tackles the problem of adapting foundation models to time-series tasks without task-specific fine-tuning by introducing an instruction-conditioned in-context learning approach, resulting in outperformance on forecasting benchmarks like fev-bench and GIFT-Eval while remaining competitive on classification and anomaly detection.
In-context learning (ICL) allows a model to adapt at inference time by conditioning on examples rather than updating parameters. Existing time-series foundation models use implicit positional context, retrieval, or task-specific objectives, but rarely explicit instruction-conditioned demonstrations. We present a foundation model for instruction-conditioned in-context time-series tasks based on a quantile-regression T5 encoder-decoder. Historical examples and queries are encoded with a structured tokenization scheme that marks target series, covariates, context, and task-specific future information. A hierarchical Transformer with per-example encoding, example-level fusion, and cross-example attention conditions decoding on demonstration pairs, enabling forecasting and related tasks without task-specific fine-tuning. We train on large-scale real and synthetic time series using supervised forecasting plus self-supervised tasks, including imputation, reconstruction, classification, anomaly detection, and source demixing. This multi-task training learns a distribution over task mappings and improves adaptation to local structure at inference time. Across diverse datasets, frequencies, and horizons, our method outperforms strong foundation baselines on point and probabilistic forecasting benchmarks, including fev-bench and GIFT-Eval, while remaining competitive on classification and anomaly detection.