Intra-Source Style Augmentation for Improved Domain Generalization
This addresses the challenge of domain shifts in applications like autonomous driving, offering a model-agnostic method that is complementary to existing techniques, though it is incremental as it builds on StyleGAN2 inversion.
The paper tackles domain generalization in semantic segmentation by proposing intra-source style augmentation (ISSA), which uses a masked noise encoder with StyleGAN2 inversion to randomize style and content in training data, achieving up to 12.4% mIoU improvements on driving-scene datasets under various data shifts.
The generalization with respect to domain shifts, as they frequently appear in applications such as autonomous driving, is one of the remaining big challenges for deep learning models. Therefore, we propose an intra-source style augmentation (ISSA) method to improve domain generalization in semantic segmentation. Our method is based on a novel masked noise encoder for StyleGAN2 inversion. The model learns to faithfully reconstruct the image preserving its semantic layout through noise prediction. Random masking of the estimated noise enables the style mixing capability of our model, i.e. it allows to alter the global appearance without affecting the semantic layout of an image. Using the proposed masked noise encoder to randomize style and content combinations in the training set, ISSA effectively increases the diversity of training data and reduces spurious correlation. As a result, we achieve up to $12.4\%$ mIoU improvements on driving-scene semantic segmentation under different types of data shifts, i.e., changing geographic locations, adverse weather conditions, and day to night. ISSA is model-agnostic and straightforwardly applicable with CNNs and Transformers. It is also complementary to other domain generalization techniques, e.g., it improves the recent state-of-the-art solution RobustNet by $3\%$ mIoU in Cityscapes to Dark Zürich.