Soft Contrastive Learning for Time Series
This work addresses representation learning for time series data, which is incremental as it modifies existing contrastive methods with soft assignments.
The paper tackles the problem of contrastive learning for time series by addressing the issue of ignoring inherent correlations when contrasting similar instances or adjacent timestamps, which deteriorates representation quality. The result is SoftCLT, a soft contrastive learning strategy that improves performance in downstream tasks like classification and anomaly detection, achieving state-of-the-art results.
Contrastive learning has shown to be effective to learn representations from time series in a self-supervised way. However, contrasting similar time series instances or values from adjacent timestamps within a time series leads to ignore their inherent correlations, which results in deteriorating the quality of learned representations. To address this issue, we propose SoftCLT, a simple yet effective soft contrastive learning strategy for time series. This is achieved by introducing instance-wise and temporal contrastive loss with soft assignments ranging from zero to one. Specifically, we define soft assignments for 1) instance-wise contrastive loss by the distance between time series on the data space, and 2) temporal contrastive loss by the difference of timestamps. SoftCLT is a plug-and-play method for time series contrastive learning that improves the quality of learned representations without bells and whistles. In experiments, we demonstrate that SoftCLT consistently improves the performance in various downstream tasks including classification, semi-supervised learning, transfer learning, and anomaly detection, showing state-of-the-art performance. Code is available at this repository: https://github.com/seunghan96/softclt.