An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
This addresses domain shift issues in semantic segmentation for applications like autonomous driving, but it is incremental as it builds on existing adversarial alignment methods.
The paper tackles the problem of unsupervised domain adaptation for semantic segmentation, where adversarial alignment often fails for tail classes or small objects, by proposing a method that trains a domain-invariant classifier using pointwise adversarial perturbations, achieving state-of-the-art performance on GTA5 -> Cityscapes and SYNTHIA -> Cityscapes tasks.
We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 -> Cityscapes and SYNTHIA -> Cityscapes.