DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation
This addresses the costly data annotation issue for 3D semantic segmentation in robotics and AR/VR by enabling effective sim-to-real adaptation, though it is incremental as it builds on existing UDA methods.
The paper tackles the problem of domain shift in 3D semantic segmentation when training on synthetic data and testing on real-world data, proposing DODA to mitigate gaps in patterns and context, which surpasses existing methods by over 13% on benchmarks.
Deep learning approaches achieve prominent success in 3D semantic segmentation. However, collecting densely annotated real-world 3D datasets is extremely time-consuming and expensive. Training models on synthetic data and generalizing on real-world scenarios becomes an appealing alternative, but unfortunately suffers from notorious domain shifts. In this work, we propose a Data-Oriented Domain Adaptation (DODA) framework to mitigate pattern and context gaps caused by different sensing mechanisms and layout placements across domains. Our DODA encompasses virtual scan simulation to imitate real-world point cloud patterns and tail-aware cuboid mixing to alleviate the interior context gap with a cuboid-based intermediate domain. The first unsupervised sim-to-real adaptation benchmark on 3D indoor semantic segmentation is also built on 3D-FRONT, ScanNet and S3DIS along with 7 popular Unsupervised Domain Adaptation (UDA) methods. Our DODA surpasses existing UDA approaches by over 13% on both 3D-FRONT -> ScanNet and 3D-FRONT -> S3DIS. Code is available at https://github.com/CVMI-Lab/DODA.