Large Pre-trained time series models for cross-domain Time series analysis tasks
This addresses the problem of inefficient model training for time-series analysis tasks by enabling cross-domain pre-training, though it is incremental as it adapts existing pre-training paradigms to time-series data.
The paper tackles the challenge of pre-training a foundational time-series model across heterogeneous domains by proposing LPTM, which uses adaptive segmentation to automatically identify optimal dataset-specific strategies during pre-training. It achieves superior forecasting and classification results, requiring up to 40% less data and 50% less training time compared to state-of-the-art baselines.
Large pre-trained models have been vital in recent advancements in domains like language and vision, making model training for individual downstream tasks more efficient and provide superior performance. However, tackling time-series analysis tasks usually involves designing and training a separate model from scratch leveraging training data and domain expertise specific to the task. We tackle a significant challenge for pre-training a foundational time-series model from multi-domain time-series datasets: extracting semantically useful tokenized inputs to the model across heterogenous time-series from different domains. We propose Large Pre-trained Time-series Models (LPTM) that introduces a novel method of adaptive segmentation that automatically identifies optimal dataset-specific segmentation strategy during pre-training. This enables LPTM to perform similar to or better than domain-specific state-of-art model when fine-tuned to different downstream time-series analysis tasks and under zero-shot settings. LPTM achieves superior forecasting and time-series classification results taking up to 40% less data and 50% less training time compared to state-of-art baselines. Code: www.github.com/AdityaLab/Samay