Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation
This work solves the labor-intensive annotation issue for semantic segmentation by improving generalization from synthetic to real data, though it is incremental.
The paper tackles the problem of domain adaptation for semantic segmentation by addressing the texture gap between synthetic and real data, achieving state-of-the-art performance.
Since annotating pixel-level labels for semantic segmentation is laborious, leveraging synthetic data is an attractive solution. However, due to the domain gap between synthetic domain and real domain, it is challenging for a model trained with synthetic data to generalize to real data. In this paper, considering the fundamental difference between the two domains as the texture, we propose a method to adapt to the texture of the target domain. First, we diversity the texture of synthetic images using a style transfer algorithm. The various textures of generated images prevent a segmentation model from overfitting to one specific (synthetic) texture. Then, we fine-tune the model with self-training to get direct supervision of the target texture. Our results achieve state-of-the-art performance and we analyze the properties of the model trained on the stylized dataset with extensive experiments.