RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network
It addresses the problem of semantic segmentation in unseen domains with limited data for applications like medical imaging, but it is incremental as it builds on existing cross-domain feature transformation methods.
The paper tackles cross-domain few-shot segmentation by proposing RestNet, which uses residual transformation to transfer knowledge while preserving intra-domain information, achieving state-of-the-art results on datasets like ISIC, Chest X-ray, and FSS-1000.
Cross-domain few-shot segmentation (CD-FSS) aims to achieve semantic segmentation in previously unseen domains with a limited number of annotated samples. Although existing CD-FSS models focus on cross-domain feature transformation, relying exclusively on inter-domain knowledge transfer may lead to the loss of critical intra-domain information. To this end, we propose a novel residual transformation network (RestNet) that facilitates knowledge transfer while retaining the intra-domain support-query feature information. Specifically, we propose a Semantic Enhanced Anchor Transform (SEAT) module that maps features to a stable domain-agnostic space using advanced semantics. Additionally, an Intra-domain Residual Enhancement (IRE) module is designed to maintain the intra-domain representation of the original discriminant space in the new space. We also propose a mask prediction strategy based on prototype fusion to help the model gradually learn how to segment. Our RestNet can transfer cross-domain knowledge from both inter-domain and intra-domain without requiring additional fine-tuning. Extensive experiments on ISIC, Chest X-ray, and FSS-1000 show that our RestNet achieves state-of-the-art performance. Our code will be available soon.