360$^\circ$ from a Single Camera: A Few-Shot Approach for LiDAR Segmentation
This addresses the costly labeling problem for LiDAR segmentation in autonomous driving or robotics, though it is incremental as it builds on existing few-shot and teacher-student methods.
The paper tackles the domain gap in LiDAR segmentation by proposing ImageTo360, a few-shot approach that uses an image teacher network to generate semantic predictions for LiDAR data, improving state-of-the-art results for label-efficient methods and surpassing some fully-supervised networks.
Deep learning applications on LiDAR data suffer from a strong domain gap when applied to different sensors or tasks. In order for these methods to obtain similar accuracy on different data in comparison to values reported on public benchmarks, a large scale annotated dataset is necessary. However, in practical applications labeled data is costly and time consuming to obtain. Such factors have triggered various research in label-efficient methods, but a large gap remains to their fully-supervised counterparts. Thus, we propose ImageTo360, an effective and streamlined few-shot approach to label-efficient LiDAR segmentation. Our method utilizes an image teacher network to generate semantic predictions for LiDAR data within a single camera view. The teacher is used to pretrain the LiDAR segmentation student network, prior to optional fine-tuning on 360$^\circ$ data. Our method is implemented in a modular manner on the point level and as such is generalizable to different architectures. We improve over the current state-of-the-art results for label-efficient methods and even surpass some traditional fully-supervised segmentation networks.