On The Robustness of Foundational 3D Medical Image Segmentation Models Against Imprecise Visual Prompts
This work addresses the under-explored issue of prompt robustness in medical AI, which is crucial for reliable clinical applications but is incremental in nature.
The study systematically investigates the robustness of 3D foundational models for medical image segmentation against imprecise visual prompts, revealing their reliance on shape and spatial cues and resilience to certain perturbations in multi-organ abdominal segmentation tasks.
While 3D foundational models have shown promise for promptable segmentation of medical volumes, their robustness to imprecise prompts remains under-explored. In this work, we aim to address this gap by systematically studying the effect of various controlled perturbations of dense visual prompts, that closely mimic real-world imprecision. By conducting experiments with two recent foundational models on a multi-organ abdominal segmentation task, we reveal several facets of promptable medical segmentation, especially pertaining to reliance on visual shape and spatial cues, and the extent of resilience of models towards certain perturbations. Codes are available at: https://github.com/ucsdbiag/Prompt-Robustness-MedSegFMs