Class-Balanced Pixel-Level Self-Labeling for Domain Adaptive Semantic Segmentation
This work addresses domain shift and class imbalance in semantic segmentation, which is crucial for real-world applications like autonomous driving, but it is incremental as it builds on existing self-training approaches.
The paper tackles domain adaptive semantic segmentation by proposing a class-balanced pixel-level self-labeling method that directly clusters target domain pixels and aligns class distributions, achieving significant performance improvements, especially on long-tailed categories, over state-of-the-art methods.
Domain adaptive semantic segmentation aims to learn a model with the supervision of source domain data, and produce satisfactory dense predictions on unlabeled target domain. One popular solution to this challenging task is self-training, which selects high-scoring predictions on target samples as pseudo labels for training. However, the produced pseudo labels often contain much noise because the model is biased to source domain as well as majority categories. To address the above issues, we propose to directly explore the intrinsic pixel distributions of target domain data, instead of heavily relying on the source domain. Specifically, we simultaneously cluster pixels and rectify pseudo labels with the obtained cluster assignments. This process is done in an online fashion so that pseudo labels could co-evolve with the segmentation model without extra training rounds. To overcome the class imbalance problem on long-tailed categories, we employ a distribution alignment technique to enforce the marginal class distribution of cluster assignments to be close to that of pseudo labels. The proposed method, namely Class-balanced Pixel-level Self-Labeling (CPSL), improves the segmentation performance on target domain over state-of-the-arts by a large margin, especially on long-tailed categories.