Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI
This work addresses the need for reliable segmentation in safety-critical medical workflows with inter-user variability, offering a framework to assess prompt dependence, though it is incremental in refining evaluation methods for existing models.
The paper tackled the problem of evaluating the reliability of promptable segmentation models in gynecological MRI by analyzing their sensitivity to user prompt variations, and found a strong negative correlation between prompt dependence metrics and segmentation performance. It introduced a novel formulation that disentangles prompt ambiguity from local sensitivity, providing interpretable insights into segmentation robustness.
Promptable segmentation models (e.g., the Segment Anything Models) enable generalizable, zero-shot segmentation across diverse domains. Although predictions are deterministic for a fixed image-prompt pair, the robustness of these models to variations in user prompts, referred to as prompt dependence, remains underexplored. In safety-critical workflows with substantial inter-user variability, interpretable and informative frameworks are needed to evaluate prompt dependence. In this work, we assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. We introduce the first formulation of prompt dependence that explicitly disentangles prompt ambiguity (inter-user variability) from local sensitivity (interaction imprecision), offering an interpretable view of segmentation robustness. Experiments on two female pelvic MRI datasets for uterus and bladder segmentation reveal a strong negative correlation between both metrics and segmentation performance, highlighting the value of our framework for assessing robustness. The two metrics have low mutual correlation, supporting the disentangled design of our formulation, and provide meaningful indicators of prompt-related failure modes.