Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation
This work addresses the problem of limited labeled data in medical image segmentation for healthcare applications, representing an incremental advancement by adapting foundation models to semi-supervised learning.
The paper tackles the challenge of semi-supervised medical image segmentation by proposing a cross prompting consistency method with the Segment Anything Model (CPC-SAM), which leverages SAM's prompt design to generate automatic supervisions across dual branches, resulting in over 9% Dice improvement on a breast cancer segmentation task.
Semi-supervised learning (SSL) has achieved notable progress in medical image segmentation. To achieve effective SSL, a model needs to be able to efficiently learn from limited labeled data and effectively exploiting knowledge from abundant unlabeled data. Recent developments in visual foundation models, such as the Segment Anything Model (SAM), have demonstrated remarkable adaptability with improved sample efficiency. To harness the power of foundation models for application in SSL, we propose a cross prompting consistency method with segment anything model (CPC-SAM) for semi-supervised medical image segmentation. Our method employs SAM's unique prompt design and innovates a cross-prompting strategy within a dual-branch framework to automatically generate prompts and supervisions across two decoder branches, enabling effectively learning from both scarce labeled and valuable unlabeled data. We further design a novel prompt consistency regularization, to reduce the prompt position sensitivity and to enhance the output invariance under different prompts. We validate our method on two medical image segmentation tasks. The extensive experiments with different labeled-data ratios and modalities demonstrate the superiority of our proposed method over the state-of-the-art SSL methods, with more than 9% Dice improvement on the breast cancer segmentation task.