Time Series as Images: Vision Transformer for Irregularly Sampled Time Series
This provides a universal framework for time series modeling, particularly beneficial for medical and activity domains, though it is incremental as it adapts existing vision transformers to a new data format.
The paper tackles the challenge of modeling irregularly sampled time series by converting them into line graph images and using pre-trained vision transformers for classification, achieving a 42.8% improvement in F1 score over specialized baselines in leave-sensors-out settings.
Irregularly sampled time series are increasingly prevalent, particularly in medical domains. While various specialized methods have been developed to handle these irregularities, effectively modeling their complex dynamics and pronounced sparsity remains a challenge. This paper introduces a novel perspective by converting irregularly sampled time series into line graph images, then utilizing powerful pre-trained vision transformers for time series classification in the same way as image classification. This method not only largely simplifies specialized algorithm designs but also presents the potential to serve as a universal framework for time series modeling. Remarkably, despite its simplicity, our approach outperforms state-of-the-art specialized algorithms on several popular healthcare and human activity datasets. Especially in the rigorous leave-sensors-out setting where a portion of variables is omitted during testing, our method exhibits strong robustness against varying degrees of missing observations, achieving an impressive improvement of 42.8% in absolute F1 score points over leading specialized baselines even with half the variables masked. Code and data are available at https://github.com/Leezekun/ViTST