PointInst3D: Segmenting 3D Instances by Points
This addresses the need for more robust and flexible 3D instance segmentation methods in computer vision, though it is incremental as it builds on existing per-point prediction frameworks.
The paper tackles the problem of 3D instance segmentation by proposing a fully-convolutional method that avoids clustering steps, using an Optimal Transport approach to assign target masks to points, achieving improved accuracy on ScanNet and S3DIS benchmarks.
The current state-of-the-art methods in 3D instance segmentation typically involve a clustering step, despite the tendency towards heuristics, greedy algorithms, and a lack of robustness to the changes in data statistics. In contrast, we propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. In doing so it avoids the challenges that clustering-based methods face: introducing dependencies among different tasks of the model. We find the key to its success is assigning a suitable target to each sampled point. Instead of the commonly used static or distance-based assignment strategies, we propose to use an Optimal Transport approach to optimally assign target masks to the sampled points according to the dynamic matching costs. Our approach achieves promising results on both ScanNet and S3DIS benchmarks. The proposed approach removes intertask dependencies and thus represents a simpler and more flexible 3D instance segmentation framework than other competing methods, while achieving improved segmentation accuracy.