Self-Supervised Learning of Disentangled Representations for Multivariate Time-Series
This work addresses representation learning challenges in fields like healthcare and industry, but it appears incremental as it builds on self-supervised learning methods with specific modifications.
The paper tackled the problem of learning disentangled representations for multivariate time-series data, which is challenging due to high dimensionality and lack of labels, by introducing TimeDRL, a framework that outperformed existing methods in forecasting and classification tasks.
Multivariate time-series data in fields like healthcare and industry are informative but challenging due to high dimensionality and lack of labels. Recent self-supervised learning methods excel in learning rich representations without labels but struggle with disentangled embeddings and inductive bias issues like transformation-invariance. To address these challenges, we introduce TimeDRL, a framework for multivariate time-series representation learning with dual-level disentangled embeddings. TimeDRL features: (i) disentangled timestamp-level and instance-level embeddings using a [CLS] token strategy; (ii) timestamp-predictive and instance-contrastive tasks for representation learning; and (iii) avoidance of augmentation methods to eliminate inductive biases. Experiments on forecasting and classification datasets show TimeDRL outperforms existing methods, with further validation in semi-supervised settings with limited labeled data.