Heuristic Self-Paced Learning for Domain Adaptive Semantic Segmentation under Adverse Conditions
This work addresses domain adaptation challenges for semantic segmentation in adverse conditions, offering an incremental improvement over existing methods.
The paper tackles the problem of category bias in unsupervised domain adaptation for semantic segmentation under adverse weather conditions by proposing an autonomous class scheduler based on reinforcement learning, achieving state-of-the-art performance on benchmarks like ACDC, Dark Zurich, and Nighttime Driving.
The learning order of semantic classes significantly impacts unsupervised domain adaptation for semantic segmentation, especially under adverse weather conditions. Most existing curricula rely on handcrafted heuristics (e.g., fixed uncertainty metrics) and follow a static schedule, which fails to adapt to a model's evolving, high-dimensional training dynamics, leading to category bias. Inspired by Reinforcement Learning, we cast curriculum learning as a sequential decision problem and propose an autonomous class scheduler. This scheduler consists of two components: (i) a high-dimensional state encoder that maps the model's training status into a latent space and distills key features indicative of progress, and (ii) a category-fair policy-gradient objective that ensures balanced improvement across classes. Coupled with mixed source-target supervision, the learned class rankings direct the network's focus to the most informative classes at each stage, enabling more adaptive and dynamic learning. It is worth noting that our method achieves state-of-the-art performance on three widely used benchmarks (e.g., ACDC, Dark Zurich, and Nighttime Driving) and shows generalization ability in synthetic-to-real semantic segmentation.