Novel class discovery meets foundation models for 3D semantic segmentation
This work addresses the problem of segmenting unlabelled classes in 3D point clouds for applications like autonomous driving and robotics, representing a pioneering effort in extending NCD to 3D data.
The paper tackles the novel task of Novel Class Discovery (NCD) for 3D point cloud semantic segmentation, showing that directly applying 2D methods yields suboptimal results and proposing a new approach with online clustering, uncertainty estimation, and semantic distillation, which demonstrates substantial superiority over baselines on datasets like SemanticKITTI, SemanticPOSS, and S3DIS.
The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.