Improving Time Series Encoding with Noise-Aware Self-Supervised Learning and an Efficient Encoder
This work improves time series analysis for applications like forecasting and classification by offering a more robust and efficient encoding method, though it is incremental as it builds on existing contrastive learning techniques.
The paper tackles the problem of time series representation learning by addressing inherent noise and inefficient encoders, proposing a noise-aware self-supervised training strategy and an efficient encoder architecture. The method outperforms state-of-the-art approaches in forecasting, classification, and abnormality detection, achieving top rank in over two-thirds of UCR classification datasets with 40% fewer parameters than the second-best approach.
In this work, we investigate the time series representation learning problem using self-supervised techniques. Contrastive learning is well-known in this area as it is a powerful method for extracting information from the series and generating task-appropriate representations. Despite its proficiency in capturing time series characteristics, these techniques often overlook a critical factor - the inherent noise in this type of data, a consideration usually emphasized in general time series analysis. Moreover, there is a notable absence of attention to developing efficient yet lightweight encoder architectures, with an undue focus on delivering contrastive losses. Our work address these gaps by proposing an innovative training strategy that promotes consistent representation learning, accounting for the presence of noise-prone signals in natural time series. Furthermore, we propose an encoder architecture that incorporates dilated convolution within the Inception block, resulting in a scalable and robust network with a wide receptive field. Experimental findings underscore the effectiveness of our method, consistently outperforming state-of-the-art approaches across various tasks, including forecasting, classification, and abnormality detection. Notably, our method attains the top rank in over two-thirds of the classification UCR datasets, utilizing only 40% of the parameters compared to the second-best approach. Our source code for CoInception framework is accessible at https://github.com/anhduy0911/CoInception.