Spatiotemporal Self-supervised Learning for Point Clouds in the Wild
This addresses the challenge of manual annotation for point cloud segmentation, particularly in applications like autonomous driving, but is incremental as it builds on existing SSL methods by adding temporal exploitation.
The paper tackles the problem of self-supervised learning for point cloud semantic segmentation by introducing a strategy that leverages both spatial and temporal information from LiDAR data, resulting in outperforming state-of-the-art methods on benchmarks.
Self-supervised learning (SSL) has the potential to benefit many applications, particularly those where manually annotating data is cumbersome. One such situation is the semantic segmentation of point clouds. In this context, existing methods employ contrastive learning strategies and define positive pairs by performing various augmentation of point clusters in a single frame. As such, these methods do not exploit the temporal nature of LiDAR data. In this paper, we introduce an SSL strategy that leverages positive pairs in both the spatial and temporal domain. To this end, we design (i) a point-to-cluster learning strategy that aggregates spatial information to distinguish objects; and (ii) a cluster-to-cluster learning strategy based on unsupervised object tracking that exploits temporal correspondences. We demonstrate the benefits of our approach via extensive experiments performed by self-supervised training on two large-scale LiDAR datasets and transferring the resulting models to other point cloud segmentation benchmarks. Our results evidence that our method outperforms the state-of-the-art point cloud SSL methods.