Robust 3D Scene Segmentation through Hierarchical and Learnable Part-Fusion
This addresses a specific bottleneck in 3D scene understanding for applications like autonomous driving and robotics, offering an incremental but effective enhancement to existing methods.
The paper tackles the part misclassification problem in 3D semantic segmentation by introducing Segment-Fusion, a hierarchical attention-based method that fuses semantic and instance information, resulting in performance improvements of up to 5% on ScanNet and S3DIS datasets.
3D semantic segmentation is a fundamental building block for several scene understanding applications such as autonomous driving, robotics and AR/VR. Several state-of-the-art semantic segmentation models suffer from the part misclassification problem, wherein parts of the same object are labelled incorrectly. Previous methods have utilized hierarchical, iterative methods to fuse semantic and instance information, but they lack learnability in context fusion, and are computationally complex and heuristic driven. This paper presents Segment-Fusion, a novel attention-based method for hierarchical fusion of semantic and instance information to address the part misclassifications. The presented method includes a graph segmentation algorithm for grouping points into segments that pools point-wise features into segment-wise features, a learnable attention-based network to fuse these segments based on their semantic and instance features, and followed by a simple yet effective connected component labelling algorithm to convert segment features to instance labels. Segment-Fusion can be flexibly employed with any network architecture for semantic/instance segmentation. It improves the qualitative and quantitative performance of several semantic segmentation backbones by upto 5% when evaluated on the ScanNet and S3DIS datasets.