Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning
This addresses a common challenge in real-world applications like healthcare and finance where time series data often has irregular sampling and missing values, representing an incremental improvement over existing approaches.
The paper tackles the problem of learning representations from irregularly sampled time series by proposing iTimER, a self-supervised pre-training framework that leverages reconstruction errors as learning signals, and demonstrates consistent outperformance over state-of-the-art methods in classification, interpolation, and forecasting tasks.
Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.