ProtoSAM: One-Shot Medical Image Segmentation With Foundational Models
This enables automated segmentation in medical imaging with minimal labeled data, though it is incremental as it builds on existing methods.
The authors tackled one-shot medical image segmentation by combining prototypical networks with the Segment Anything Model (SAM), achieving state-of-the-art results on multiple datasets without fine-tuning the foundation model.
This work introduces a new framework, ProtoSAM, for one-shot medical image segmentation. It combines the use of prototypical networks, known for few-shot segmentation, with SAM - a natural image foundation model. The method proposed creates an initial coarse segmentation mask using the ALPnet prototypical network, augmented with a DINOv2 encoder. Following the extraction of an initial mask, prompts are extracted, such as points and bounding boxes, which are then input into the Segment Anything Model (SAM). State-of-the-art results are shown on several medical image datasets and demonstrate automated segmentation capabilities using a single image example (one shot) with no need for fine-tuning of the foundation model. Our code is available at: https://github.com/levayz/ProtoSAM