Cluster-to-adapt: Few Shot Domain Adaptation for Semantic Segmentation across Disjoint Labels
It addresses a general scenario in semantic segmentation where categories differ across domains, which is incremental as it builds on existing domain adaptation methods.
The paper tackles domain adaptation for semantic segmentation when source and target datasets have non-overlapping label spaces, proposing Cluster-to-Adapt (C2A) to align related categories and prevent negative transfer, resulting in consistent performance improvements in few-shot and zero-shot settings.
Domain adaptation for semantic segmentation across datasets consisting of the same categories has seen several recent successes. However, a more general scenario is when the source and target datasets correspond to non-overlapping label spaces. For example, categories in segmentation datasets change vastly depending on the type of environment or application, yet share many valuable semantic relations. Existing approaches based on feature alignment or discrepancy minimization do not take such category shift into account. In this work, we present Cluster-to-Adapt (C2A), a computationally efficient clustering-based approach for domain adaptation across segmentation datasets with completely different, but possibly related categories. We show that such a clustering objective enforced in a transformed feature space serves to automatically select categories across source and target domains that can be aligned for improving the target performance, while preventing negative transfer for unrelated categories. We demonstrate the effectiveness of our approach through experiments on the challenging problem of outdoor to indoor adaptation for semantic segmentation in few-shot as well as zero-shot settings, with consistent improvements in performance over existing approaches and baselines in all cases.