Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition
This work addresses view-invariant representation learning for skeleton-based action recognition, which is an incremental improvement over existing approaches.
The paper tackles the problem of learning view-invariant representations for skeleton-based action recognition by proposing FoCoViL, a self-supervised framework that suppresses view-specific information and enhances feature discrimination. The method achieves superior recognition performance on both unsupervised and supervised classifiers, as demonstrated through extensive experiments.
Learning view-invariant representation is a key to improving feature discrimination power for skeleton-based action recognition. Existing approaches cannot effectively remove the impact of viewpoint due to the implicit view-dependent representations. In this work, we propose a self-supervised framework called Focalized Contrastive View-invariant Learning (FoCoViL), which significantly suppresses the view-specific information on the representation space where the viewpoints are coarsely aligned. By maximizing mutual information with an effective contrastive loss between multi-view sample pairs, FoCoViL associates actions with common view-invariant properties and simultaneously separates the dissimilar ones. We further propose an adaptive focalization method based on pairwise similarity to enhance contrastive learning for a clearer cluster boundary in the learned space. Different from many existing self-supervised representation learning work that rely heavily on supervised classifiers, FoCoViL performs well on both unsupervised and supervised classifiers with superior recognition performance. Extensive experiments also show that the proposed contrastive-based focalization generates a more discriminative latent representation.