Continual Unsupervised Domain Adaptation for Semantic Segmentation
This addresses a practical challenge in real-world applications where target data for domain adaptation is presented incrementally, rather than all at once, making it incremental but useful for scenarios like autonomous driving.
The paper tackles the problem of adapting semantic segmentation models to multiple target domains sequentially, where data arrives over time, by proposing an Expanding Target-specific Memory framework and a Double Hinge Adversarial loss to prevent forgetting and improve performance. The model outperforms state-of-the-art methods on benchmarks like GTA5, SYNTHIA, CityScapes, IDD, and Cross-City datasets.
Unsupervised Domain Adaptation (UDA) for semantic segmentation has been favorably applied to real-world scenarios in which pixel-level labels are hard to be obtained. In most of the existing UDA methods, all target data are assumed to be introduced simultaneously. Yet, the data are usually presented sequentially in the real world. Moreover, Continual UDA, which deals with more practical scenarios with multiple target domains in the continual learning setting, has not been actively explored. In this light, we propose Continual UDA for semantic segmentation based on a newly designed Expanding Target-specific Memory (ETM) framework. Our novel ETM framework contains Target-specific Memory (TM) for each target domain to alleviate catastrophic forgetting. Furthermore, a proposed Double Hinge Adversarial (DHA) loss leads the network to produce better UDA performance overall. Our design of the TM and training objectives let the semantic segmentation network adapt to the current target domain while preserving the knowledge learned on previous target domains. The model with the proposed framework outperforms other state-of-the-art models in continual learning settings on standard benchmarks such as GTA5, SYNTHIA, CityScapes, IDD, and Cross-City datasets. The source code is available at https://github.com/joonh-kim/ETM.