PointMatch: A Consistency Training Framework for Weakly Supervised Semantic Segmentation of 3D Point Clouds
This addresses the costly annotation problem for 3D point cloud segmentation, offering a robust solution for applications like robotics and autonomous driving, though it is incremental over existing weakly supervised methods.
The paper tackles the problem of weakly supervised semantic segmentation of 3D point clouds by proposing PointMatch, a framework that uses consistency regularization to leverage data information and weak labels, achieving state-of-the-art performance with improvements of up to 21.2% on sparse label settings.
Semantic segmentation of point cloud usually relies on dense annotation that is exhausting and costly, so it attracts wide attention to investigate solutions for the weakly supervised scheme with only sparse points annotated. Existing works start from the given labels and propagate them to highly-related but unlabeled points, with the guidance of data, e.g. intra-point relation. However, it suffers from (i) the inefficient exploitation of data information, and (ii) the strong reliance on labels thus is easily suppressed when given much fewer annotations. Therefore, we propose a novel framework, PointMatch, that stands on both data and label, by applying consistency regularization to sufficiently probe information from data itself and leveraging weak labels as assistance at the same time. By doing so, meaningful information can be learned from both data and label for better representation learning, which also enables the model more robust to the extent of label sparsity. Simple yet effective, the proposed PointMatch achieves the state-of-the-art performance under various weakly-supervised schemes on both ScanNet-v2 and S3DIS datasets, especially on the settings with extremely sparse labels, e.g. surpassing SQN by 21.2% and 17.2% on the 0.01% and 0.1% setting of ScanNet-v2, respectively.