ProCST: Boosting Semantic Segmentation Using Progressive Cyclic Style-Transfer
This work addresses the costly annotation problem in computer vision by enhancing domain adaptation for semantic segmentation, though it is incremental as it builds on existing UDA techniques.
The authors tackled the domain gap between synthetic and real-world data in semantic segmentation by proposing a two-stage framework that first generates 'Source in Target' (SiT) data via progressive style-transfer, then applies standard UDA methods. They demonstrated improvements with three state-of-the-art UDA methods on tasks like GTA5 to Cityscapes, achieving competitive performance gains.
Using synthetic data for training neural networks that achieve good performance on real-world data is an important task as it can reduce the need for costly data annotation. Yet, synthetic and real world data have a domain gap. Reducing this gap, also known as domain adaptation, has been widely studied in recent years. Closing the domain gap between the source (synthetic) and target (real) data by directly performing the adaptation between the two is challenging. In this work, we propose a novel two-stage framework for improving domain adaptation techniques on image data. In the first stage, we progressively train a multi-scale neural network to perform image translation from the source domain to the target domain. We denote the new transformed data as "Source in Target" (SiT). Then, we insert the generated SiT data as the input to any standard UDA approach. This new data has a reduced domain gap from the desired target domain, which facilitates the applied UDA approach to close the gap further. We emphasize the effectiveness of our method via a comparison to other leading UDA and image-to-image translation techniques when used as SiT generators. Moreover, we demonstrate the improvement of our framework with three state-of-the-art UDA methods for semantic segmentation, HRDA, DAFormer and ProDA, on two UDA tasks, GTA5 to Cityscapes and Synthia to Cityscapes.