Learnable Ophthalmology SAM
This work addresses the problem of limited generalization in ophthalmology image segmentation for medical practitioners, offering an incremental improvement by adapting a foundational CV model to the medical domain.
The paper tackles the challenge of applying segmentation algorithms to diverse ophthalmology images by proposing a learnable prompt layer for Segment Anything (SAM), achieving effective multi-target segmentation across nine datasets with a one-shot training approach.
Segmentation is vital for ophthalmology image analysis. But its various modal images hinder most of the existing segmentation algorithms applications, as they rely on training based on a large number of labels or hold weak generalization ability. Based on Segment Anything (SAM), we propose a simple but effective learnable prompt layer suitable for multiple target segmentation in ophthalmology multi-modal images, named Learnable Ophthalmology Segment Anything (SAM). The learnable prompt layer learns medical prior knowledge from each transformer layer. During training, we only train the prompt layer and task head based on a one-shot mechanism. We demonstrate the effectiveness of our thought based on four medical segmentation tasks based on nine publicly available datasets. Moreover, we only provide a new improvement thought for applying the existing fundamental CV models in the medical field. Our codes are available at \href{https://github.com/Qsingle/LearnablePromptSAM}{website}.