eMargin: Revisiting Contrastive Learning with Margin-Based Separation
This work addresses representation learning for time series data, but it is incremental as it modifies an existing contrastive learning framework with limited impact on practical applications.
The paper tackled the problem of improving contrastive learning for time series representation by introducing an adaptive margin (eMargin) into the loss function, finding that it outperformed state-of-the-art baselines in unsupervised clustering metrics but struggled in downstream classification tasks.
We revisit previous contrastive learning frameworks to investigate the effect of introducing an adaptive margin into the contrastive loss function for time series representation learning. Specifically, we explore whether an adaptive margin (eMargin), adjusted based on a predefined similarity threshold, can improve the separation between adjacent but dissimilar time steps and subsequently lead to better performance in downstream tasks. Our study evaluates the impact of this modification on clustering performance and classification in three benchmark datasets. Our findings, however, indicate that achieving high scores on unsupervised clustering metrics does not necessarily imply that the learned embeddings are meaningful or effective in downstream tasks. To be specific, eMargin added to InfoNCE consistently outperforms state-of-the-art baselines in unsupervised clustering metrics, but struggles to achieve competitive results in downstream classification with linear probing. The source code is publicly available at https://github.com/sfi-norwai/eMargin.