Instance-Aware Test-Time Segmentation for Continual Domain Shifts
This work addresses the challenge of reliable semantic segmentation under continual domain shifts, which is crucial for applications like autonomous driving, but it is incremental as it builds on existing test-time adaptation methods.
The paper tackled the problem of adapting pre-trained models to continuously evolving domains in semantic segmentation, where existing methods fail to account for varying difficulty across classes and instances, and proposed an approach that adaptively adjusts pseudo labels and dynamically balances learning, resulting in consistent outperformance of state-of-the-art techniques across eight scenarios.
Continual Test-Time Adaptation (CTTA) enables pre-trained models to adapt to continuously evolving domains. Existing methods have improved robustness but typically rely on fixed or batch-level thresholds, which cannot account for varying difficulty across classes and instances. This limitation is especially problematic in semantic segmentation, where each image requires dense, multi-class predictions. We propose an approach that adaptively adjusts pseudo labels to reflect the confidence distribution within each image and dynamically balances learning toward classes most affected by domain shifts. This fine-grained, class- and instance-aware adaptation produces more reliable supervision and mitigates error accumulation throughout continual adaptation. Extensive experiments across eight CTTA and TTA scenarios, including synthetic-to-real and long-term shifts, show that our method consistently outperforms state-of-the-art techniques, setting a new standard for semantic segmentation under evolving conditions.