Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation
This addresses the challenge of limited annotated data in medical imaging for clinicians and researchers, though it appears incremental as it builds on existing SAM and optimization techniques.
The researchers tackled the problem of reducing annotation requirements for medical image segmentation by enhancing the Segment Anything Model (SAM) with unsupervised prompting and preference optimization, achieving state-of-the-art performance across multiple modalities like X-ray and CT scans.
Foundational models such as the Segment Anything Model (SAM) are gaining traction in medical imaging segmentation, supporting multiple downstream tasks. However, such models are supervised in nature, still relying on large annotated datasets or prompts supplied by experts. Conventional techniques such as active learning to alleviate such limitations are limited in scope and still necessitate continuous human involvement and complex domain knowledge for label refinement or establishing reward ground truth. To address these challenges, we propose an enhanced Segment Anything Model (SAM) framework that utilizes annotation-efficient prompts generated in a fully unsupervised fashion, while still capturing essential semantic, location, and shape information through contrastive language-image pretraining and visual question answering. We adopt the direct preference optimization technique to design an optimal policy that enables the model to generate high-fidelity segmentations with simple ratings or rankings provided by a virtual annotator simulating the human annotation process. State-of-the-art performance of our framework in tasks such as lung segmentation, breast tumor segmentation, and organ segmentation across various modalities, including X-ray, ultrasound, and abdominal CT, justifies its effectiveness in low-annotation data scenarios.