ClickSeg: 3D Instance Segmentation with Click-Level Weak Annotations
This addresses the costly annotation problem for 3D instance segmentation in computer vision, offering a significant reduction in labeling effort while maintaining high performance.
The paper tackles the problem of reducing annotation costs for 3D instance segmentation by proposing ClickSeg, a method that uses only one point per instance as weak supervision, achieving a +9.4% mAP improvement on ScanNetV2 and reaching ~90% of fully-supervised accuracy with 0.02% supervision.
3D instance segmentation methods often require fully-annotated dense labels for training, which are costly to obtain. In this paper, we present ClickSeg, a novel click-level weakly supervised 3D instance segmentation method that requires one point per instance annotation merely. Such a problem is very challenging due to the extremely limited labels, which has rarely been solved before. We first develop a baseline weakly-supervised training method, which generates pseudo labels for unlabeled data by the model itself. To utilize the property of click-level annotation setting, we further propose a new training framework. Instead of directly using the model inference way, i.e., mean-shift clustering, to generate the pseudo labels, we propose to use k-means with fixed initial seeds: the annotated points. New similarity metrics are further designed for clustering. Experiments on ScanNetV2 and S3DIS datasets show that the proposed ClickSeg surpasses the previous best weakly supervised instance segmentation result by a large margin (e.g., +9.4% mAP on ScanNetV2). Using 0.02% supervision signals merely, ClickSeg achieves $\sim$90% of the accuracy of the fully-supervised counterpart. Meanwhile, it also achieves state-of-the-art semantic segmentation results among weakly supervised methods that use the same annotation settings.