SAMDA: Leveraging SAM on Few-Shot Domain Adaptation for Electronic Microscopy Segmentation
This work addresses domain adaptation for medical image segmentation, offering a solution for scenarios with scarce annotations, though it is incremental as it builds on existing models like SAM and nnUNet.
The paper tackles the problem of limited transferability in electronic microscopy segmentation with few samples by proposing SAMDA, a framework combining SAM and nnUNet, which improves the dice coefficient by 6.7% on target domains and outperforms 10-shot adaptation with a single annotated image.
It has been shown that traditional deep learning methods for electronic microscopy segmentation usually suffer from low transferability when samples and annotations are limited, while large-scale vision foundation models are more robust when transferring between different domains but facing sub-optimal improvement under fine-tuning. In this work, we present a new few-shot domain adaptation framework SAMDA, which combines the Segment Anything Model(SAM) with nnUNet in the embedding space to achieve high transferability and accuracy. Specifically, we choose the Unet-based network as the "expert" component to learn segmentation features efficiently and design a SAM-based adaptation module as the "generic" component for domain transfer. By amalgamating the "generic" and "expert" components, we mitigate the modality imbalance in the complex pre-training knowledge inherent to large-scale Vision Foundation models and the challenge of transferability inherent to traditional neural networks. The effectiveness of our model is evaluated on two electron microscopic image datasets with different modalities for mitochondria segmentation, which improves the dice coefficient on the target domain by 6.7%. Also, the SAM-based adaptor performs significantly better with only a single annotated image than the 10-shot domain adaptation on nnUNet. We further verify our model on four MRI datasets from different sources to prove its generalization ability.