Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos
This addresses the challenge of capturing fine-grained semantics in dynamic point clouds for applications in computer vision, though it appears incremental as it builds on existing contrastive learning approaches.
The paper tackles the problem of self-supervised learning on point cloud videos by proposing a framework that conducts contrastive learning at the point level and uses semantic clustering, outperforming supervised methods on various downstream tasks.
We propose a unified point cloud video self-supervised learning framework for object-centric and scene-centric data. Previous methods commonly conduct representation learning at the clip or frame level and cannot well capture fine-grained semantics. Instead of contrasting the representations of clips or frames, in this paper, we propose a unified self-supervised framework by conducting contrastive learning at the point level. Moreover, we introduce a new pretext task by achieving semantic alignment of superpoints, which further facilitates the representations to capture semantic cues at multiple scales. In addition, due to the high redundancy in the temporal dimension of dynamic point clouds, directly conducting contrastive learning at the point level usually leads to massive undesired negatives and insufficient modeling of positive representations. To remedy this, we propose a selection strategy to retain proper negatives and make use of high-similarity samples from other instances as positive supplements. Extensive experiments show that our method outperforms supervised counterparts on a wide range of downstream tasks and demonstrates the superior transferability of the learned representations.