Contrastive Self-Supervised Learning for Skeleton Representations
This work addresses the challenge of representation learning for skeleton data in computer vision, but it is incremental as it applies an existing method to a specific domain with systematic evaluations.
The paper tackles the problem of learning effective representations for human skeleton point clouds by applying SimCLR, a contrastive self-supervised learning method, and systematically evaluates algorithmic decisions like augmentations and dataset partitioning. The result shows that combining spatial and temporal augmentations, using additional datasets, and employing a graph neural network encoder improves performance on downstream tasks such as skeleton reconstruction, motion prediction, and activity classification, with pre-training on over 40 million skeleton frames.
Human skeleton point clouds are commonly used to automatically classify and predict the behaviour of others. In this paper, we use a contrastive self-supervised learning method, SimCLR, to learn representations that capture the semantics of skeleton point clouds. This work focuses on systematically evaluating the effects that different algorithmic decisions (including augmentations, dataset partitioning and backbone architecture) have on the learned skeleton representations. To pre-train the representations, we normalise six existing datasets to obtain more than 40 million skeleton frames. We evaluate the quality of the learned representations with three downstream tasks: skeleton reconstruction, motion prediction, and activity classification. Our results demonstrate the importance of 1) combining spatial and temporal augmentations, 2) including additional datasets for encoder training, and 3) and using a graph neural network as an encoder.