Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Model
This work addresses the problem of enhancing medical image segmentation for healthcare applications by leveraging a pre-trained foundation model, though it is incremental as it builds on existing methods.
The paper tackles the challenge of adapting the Segment Anything Model (SAM), a general-purpose segmentation foundation model, for medical image segmentation by using its outputs to augment inputs for existing models like U-Net, resulting in improved performance across three segmentation tasks.
The Segment Anything Model (SAM) is a recently developed large model for general-purpose segmentation for computer vision tasks. SAM was trained using 11 million images with over 1 billion masks and can produce segmentation results for a wide range of objects in natural scene images. SAM can be viewed as a general perception model for segmentation (partitioning images into semantically meaningful regions). Thus, how to utilize such a large foundation model for medical image segmentation is an emerging research target. This paper shows that although SAM does not immediately give high-quality segmentation for medical image data, its generated masks, features, and stability scores are useful for building and training better medical image segmentation models. In particular, we demonstrate how to use SAM to augment image input for commonly-used medical image segmentation models (e.g., U-Net). Experiments on three segmentation tasks show the effectiveness of our proposed SAMAug method. The code is available at \url{https://github.com/yizhezhang2000/SAMAug}.