Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series
This work addresses the problem of costly labeling in time series data, particularly for medical applications, by providing an incremental improvement in unsupervised representation learning.
The paper tackles the challenge of learning useful representations from time series data without labels by proposing an unsupervised contrastive learning framework that uses a novel loss based on mixing data samples and predicting the mixing component. Experiments show it outperforms other representation learning methods on univariate and multivariate time series and benefits transfer learning for clinical applications.
The lack of labeled data is a key challenge for learning useful representation from time series data. However, an unsupervised representation framework that is capable of producing high quality representations could be of great value. It is key to enabling transfer learning, which is especially beneficial for medical applications, where there is an abundance of data but labeling is costly and time consuming. We propose an unsupervised contrastive learning framework that is motivated from the perspective of label smoothing. The proposed approach uses a novel contrastive loss that naturally exploits a data augmentation scheme in which new samples are generated by mixing two data samples with a mixing component. The task in the proposed framework is to predict the mixing component, which is utilized as soft targets in the loss function. Experiments demonstrate the framework's superior performance compared to other representation learning approaches on both univariate and multivariate time series and illustrate its benefits for transfer learning for clinical time series.