Self-Supervised Pre-training of 3D Point Cloud Networks with Image Data
This work addresses the challenge of costly annotations in 3D datasets, offering an incremental improvement by leveraging existing image data to simplify pre-training.
The paper tackles the problem of reducing annotation requirements for 3D semantic segmentation by proposing a self-supervised pre-training method that uses image data to train 3D models, achieving comparable performance to multi-scan, point cloud-only methods.
Reducing the quantity of annotations required for supervised training is vital when labels are scarce and costly. This reduction is especially important for semantic segmentation tasks involving 3D datasets that are often significantly smaller and more challenging to annotate than their image-based counterparts. Self-supervised pre-training on large unlabelled datasets is one way to reduce the amount of manual annotations needed. Previous work has focused on pre-training with point cloud data exclusively; this approach often requires two or more registered views. In the present work, we combine image and point cloud modalities, by first learning self-supervised image features and then using these features to train a 3D model. By incorporating image data, which is often included in many 3D datasets, our pre-training method only requires a single scan of a scene. We demonstrate that our pre-training approach, despite using single scans, achieves comparable performance to other multi-scan, point cloud-only methods.