Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation
This work addresses the underexplored area of unsupervised skeleton-based action segmentation, offering a solution for real-world applications where data annotation is costly, though it is incremental as it builds on existing unsupervised techniques.
The paper tackles the problem of unsupervised temporal action segmentation from skeleton sequences, which avoids the need for expensive annotated data, and demonstrates that their method outperforms current state-of-the-art unsupervised methods on three widely used datasets.
Current state-of-the-art methods for skeleton-based temporal action segmentation are predominantly supervised and require annotated data, which is expensive to collect. In contrast, existing unsupervised temporal action segmentation methods have focused primarily on video data, while skeleton sequences remain underexplored, despite their relevance to real-world applications, robustness, and privacy-preserving nature. In this paper, we propose a novel approach for unsupervised skeleton-based temporal action segmentation. Our method utilizes a sequence-to-sequence temporal autoencoder that keeps the information of the different joints disentangled in the embedding space. Latent skeleton sequences are then divided into non-overlapping patches and quantized to obtain distinctive skeleton motion words, driving the discovery of semantically meaningful action clusters. We thoroughly evaluate the proposed approach on three widely used skeleton-based datasets, namely HuGaDB, LARa, and BABEL. The results demonstrate that our model outperforms the current state-of-the-art unsupervised temporal action segmentation methods. Code is available at https://github.com/bachlab/SMQ .