CoLES: Contrastive Learning for Event Sequences with Self-Supervision
This work addresses the problem of learning representations from user-generated event sequences for financial services and other domains, offering a novel application with substantial impact.
The paper tackled the problem of self-supervised learning for discrete event sequences, proposing CoLES, a method that adapts contrastive learning to this domain, resulting in significant performance improvements on downstream tasks and financial gains measured in hundreds of millions of dollars yearly.
We address the problem of self-supervised learning on discrete event sequences generated by real-world users. Self-supervised learning incorporates complex information from the raw data in low-dimensional fixed-length vector representations that could be easily applied in various downstream machine learning tasks. In this paper, we propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains, to the discrete event sequences domain in a self-supervised setting. We deployed CoLES embeddings based on sequences of transactions at the large European financial services company. Usage of CoLES embeddings significantly improves the performance of the pre-existing models on downstream tasks and produces significant financial gains, measured in hundreds of millions of dollars yearly. We also evaluated CoLES on several public event sequences datasets and showed that CoLES representations consistently outperform other methods on different downstream tasks.