Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation
This addresses the limitation of few-shot segmentation for real-world use cases where domains differ from training, though it appears incremental as it builds on existing cross-domain few-shot segmentation frameworks.
The paper tackles the problem of few-shot segmentation performance declining when facing images from different domains than training, showing that test-time task-adaption with small networks appended to a classification-pretrained backbone and consistency guidance outperforms existing approaches while eliminating training and main segmentation networks, achieving new state-of-the-art performance in cross-domain few-shot segmentation.
Few-shot segmentation performance declines substantially when facing images from a domain different than the training domain, effectively limiting real-world use cases. To alleviate this, recently cross-domain few-shot segmentation (CD-FSS) has emerged. Works that address this task mainly attempted to learn segmentation on a source domain in a manner that generalizes across domains. Surprisingly, we can outperform these approaches while eliminating the training stage and removing their main segmentation network. We show test-time task-adaption is the key for successful CD-FSS instead. Task-adaption is achieved by appending small networks to the feature pyramid of a conventionally classification-pretrained backbone. To avoid overfitting to the few labeled samples in supervised fine-tuning, consistency across augmented views of input images serves as guidance while learning the parameters of the attached layers. Despite our self-restriction not to use any images other than the few labeled samples at test time, we achieve new state-of-the-art performance in CD-FSS, evidencing the need to rethink approaches for the task.