COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation
This addresses the annotation bottleneck for 3D segmentation in applications like autonomous driving, though it is incremental as it builds on existing contrastive learning and weak supervision techniques.
The paper tackles the problem of costly 3D point cloud annotation by proposing COARSE3D, a weakly-supervised contrastive learning method that reduces annotation needs to as low as 0.001% and outperforms baselines on three real-world outdoor datasets.
Annotation of large-scale 3D data is notoriously cumbersome and costly. As an alternative, weakly-supervised learning alleviates such a need by reducing the annotation by several order of magnitudes. We propose COARSE3D, a novel architecture-agnostic contrastive learning strategy for 3D segmentation. Since contrastive learning requires rich and diverse examples as keys and anchors, we leverage a prototype memory bank capturing class-wise global dataset information efficiently into a small number of prototypes acting as keys. An entropy-driven sampling technique then allows us to select good pixels from predictions as anchors. Experiments on three projection-based backbones show we outperform baselines on three challenging real-world outdoor datasets, working with as low as 0.001% annotations.