Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition
This work addresses the problem of data annotation scarcity in human activity recognition for ubiquitous computing, though it is incremental as it builds on existing SSL methods.
The paper tackles the challenge of requiring large annotated datasets for Human Activity Recognition by upgrading a contrastive self-supervised learning framework with a temporal feature alignment procedure using dynamic time warping, resulting in robust feature representations that outperform recent SSL baselines and supervised models in semi-supervised learning.
Automated Human Activity Recognition has long been a problem of great interest in human-centered and ubiquitous computing. In the last years, a plethora of supervised learning algorithms based on deep neural networks has been suggested to address this problem using various modalities. While every modality has its own limitations, there is one common challenge. Namely, supervised learning requires vast amounts of annotated data which is practically hard to collect. In this paper, we benefit from the self-supervised learning paradigm (SSL) that is typically used to learn deep feature representations from unlabeled data. Moreover, we upgrade a contrastive SSL framework, namely SimCLR, widely used in various applications by introducing a temporal feature alignment procedure for Human Activity Recognition. Specifically, we propose integrating a dynamic time warping (DTW) algorithm in a latent space to force features to be aligned in a temporal dimension. Extensive experiments have been conducted for the unimodal scenario with inertial modality as well as in multimodal settings using inertial and skeleton data. According to the obtained results, the proposed approach has a great potential in learning robust feature representations compared to the recent SSL baselines, and clearly outperforms supervised models in semi-supervised learning. The code for the unimodal case is available via the following link: https://github.com/bulatkh/csshar_tfa.