Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor Point Clouds
This addresses the annotation cost and domain shift issue for 3D indoor scene understanding, but it is incremental as it builds on existing domain generalization methods.
The paper tackles the problem of semantic segmentation in 3D indoor point clouds suffering from performance degradation due to domain gaps between synthetic training and real-world testing data, proposing a clustering instance mix augmentation and multi-prototypes to improve generalization, with experiments showing improved performance on real-world datasets.
Semantic segmentation in 3D indoor scenes has achieved remarkable performance under the supervision of large-scale annotated data. However, previous works rely on the assumption that the training and testing data are of the same distribution, which may suffer from performance degradation when evaluated on the out-of-distribution scenes. To alleviate the annotation cost and the performance degradation, this paper introduces the synthetic-to-real domain generalization setting to this task. Specifically, the domain gap between synthetic and real-world point cloud data mainly lies in the different layouts and point patterns. To address these problems, we first propose a clustering instance mix (CINMix) augmentation technique to diversify the layouts of the source data. In addition, we augment the point patterns of the source data and introduce non-parametric multi-prototypes to ameliorate the intra-class variance enlarged by the augmented point patterns. The multi-prototypes can model the intra-class variance and rectify the global classifier in both training and inference stages. Experiments on the synthetic-to-real benchmark demonstrate that both CINMix and multi-prototypes can narrow the distribution gap and thus improve the generalization ability on real-world datasets.