VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel
This work addresses vessel segmentation for clinical diagnosis and planning, offering an incremental improvement over existing SAM-based methods.
The paper tackles the problem of accurate vessel segmentation for clinical applications by introducing VesSAM, a framework that integrates a convolutional adapter, multi-prompt encoder, and lightweight mask decoder, resulting in over 10% Dice and 13% IoU improvements over state-of-the-art PEFT-based SAM variants and competitive performance with fully fine-tuned methods using fewer parameters.
Accurate vessel segmentation is critical for clinical applications such as disease diagnosis and surgical planning, yet remains challenging due to thin, branching structures and low texture contrast. While foundation models like the Segment Anything Model (SAM) have shown promise in generic segmentation, they perform sub-optimally on vascular structures. In this work, we present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation. VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, including skeletons, bifurcation points, and segment midpoints, via hierarchical cross-attention, and (3) a lightweight mask decoder to reduce jagged artifacts. We also introduce an automated pipeline to generate structured multi-prompt annotations, and curate a diverse benchmark dataset spanning 8 datasets across 5 imaging modalities. Experimental results demonstrate that VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU, and achieves competitive performance compared to fully fine-tuned methods, with significantly fewer parameters. VesSAM also generalizes well to out-of-distribution (OoD) settings, outperforming all baselines in average OoD Dice and IoU.