Jose L. Gomez

Yachan Guo, Yi Xiao, Danna Xue et al.

Instance segmentation is crucial for autonomous driving, but is hindered by the lack of annotated real-world data due to expensive labeling costs. Unsupervised Domain Adaptation (UDA) offers a solution by transferring knowledge from labeled synthetic data to unlabeled real-world data. While UDA methods for synthetic to real-world domains (synth-to-real) excel in tasks such as semantic segmentation and object detection, their application to instance segmentation for autonomous driving remains underexplored and often relies on suboptimal baselines. We introduce UDA4Inst, a powerful framework for synth-to-real UDA in instance segmentation. Our framework enhances instance segmentation through Semantic Category Training and Bidirectional Mixing Training. Semantic Category Training groups semantically related classes for separate training, improving pseudo-label quality and segmentation accuracy. Bidirectional Mixing Training combines instance-wise and patch-wise data mixing, creating coherent composites that enhance generalization across domains. Extensive experiments show UDA4Inst sets a new state-of-the-art on the SYNTHIA-> Cityscapes benchmark (mAP 31.3) and introduces results on novel datasets, using UrbanSyn and Synscapes as sources and Cityscapes and KITTI360 as targets. Code and models are available at https://github.com/gyc-code/UDA4Inst.

Antonio M. Lopez, Jiaolong Xu, Jose L. Gomez et al.

Supervised learning tends to produce more accurate classifiers than unsupervised learning in general. This implies that training data is preferred with annotations. When addressing visual perception challenges, such as localizing certain object classes within an image, the learning of the involved classifiers turns out to be a practical bottleneck. The reason is that, at least, we have to frame object examples with bounding boxes in thousands of images. A priori, the more complex the model is regarding its number of parameters, the more annotated examples are required. This annotation task is performed by human oracles, which ends up in inaccuracies and errors in the annotations (aka ground truth) since the task is inherently very cumbersome and sometimes ambiguous. As an alternative we have pioneered the use of virtual worlds for collecting such annotations automatically and with high precision. However, since the models learned with virtual data must operate in the real world, we still need to perform domain adaptation (DA). In this chapter we revisit the DA of a deformable part-based model (DPM) as an exemplifying case of virtual- to-real-world DA. As a use case, we address the challenge of vehicle detection for driver assistance, using different publicly available virtual-world data. While doing so, we investigate questions such as: how does the domain gap behave due to virtual-vs-real data with respect to dominant object appearance per domain, as well as the role of photo-realism in the virtual world.