Towards Interpretable Time Series Foundation Models
This work addresses the need for interpretable time series analysis in domains like on-device or privacy-sensitive applications, though it is incremental as it builds on existing methods for distillation and synthetic data generation.
The paper tackled the problem of building interpretable time series foundation models by distilling reasoning capabilities into small language models, showing that post-trained models acquire meaningful interpretive capabilities for trend direction, noise intensity, and extremum localization, making them suitable for on-device or privacy-sensitive deployment.
In this paper, we investigate the distillation of time series reasoning capabilities into small, instruction-tuned language models as a step toward building interpretable time series foundation models. Leveraging a synthetic dataset of mean-reverting time series with systematically varied trends and noise levels, we generate natural language annotations using a large multimodal model and use these to supervise the fine-tuning of compact Qwen models. We introduce evaluation metrics that assess the quality of the distilled reasoning - focusing on trend direction, noise intensity, and extremum localization - and show that the post-trained models acquire meaningful interpretive capabilities. Our results highlight the feasibility of compressing time series understanding into lightweight, language-capable models suitable for on-device or privacy-sensitive deployment. This work contributes a concrete foundation toward developing small, interpretable models that explain temporal patterns in natural language.