From Specialist to Generalist: Unlocking SAM's Learning Potential on Unlabeled Medical Images
This addresses label-efficient medical image segmentation for healthcare applications, offering an incremental improvement by leveraging existing models in a novel cooperative setup.
The paper tackled adapting the Segment Anything Model (SAM) to medical images with limited labels by introducing SC-SAM, a specialist-generalist framework that uses U-Net and SAM in a bidirectional co-training loop, achieving state-of-the-art results on prostate MRI and polyp segmentation benchmarks.
Foundation models like the Segment Anything Model (SAM) show strong generalization, yet adapting them to medical images remains difficult due to domain shift, scarce labels, and the inability of Parameter-Efficient Fine-Tuning (PEFT) to exploit unlabeled data. While conventional models like U-Net excel in semi-supervised medical learning, their potential to assist a PEFT SAM has been largely overlooked. We introduce SC-SAM, a specialist-generalist framework where U-Net provides point-based prompts and pseudo-labels to guide SAM's adaptation, while SAM serves as a powerful generalist supervisor to regularize U-Net. This reciprocal guidance forms a bidirectional co-training loop that allows both models to effectively exploit the unlabeled data. Across prostate MRI and polyp segmentation benchmarks, our method achieves state-of-the-art results, outperforming other existing semi-supervised SAM variants and even medical foundation models like MedSAM, highlighting the value of specialist-generalist cooperation for label-efficient medical image segmentation. Our code is available at https://github.com/vnlvi2k3/SC-SAM.