Rethinking Uncertainty in Segmentation: From Estimation to Decision
For practitioners of medical image segmentation, this work provides a practical framework to convert uncertainty estimates into actionable deferral policies, demonstrating that decision-oriented evaluation is more meaningful than standard uncertainty metrics.
The paper addresses the gap between uncertainty estimation and decision-making in medical image segmentation, showing that optimizing uncertainty alone misses most safety gains. Their confidence-aware deferral rule removes up to 80% of errors at 25% pixel deferral across retinal vessel segmentation benchmarks.
In medical image segmentation, uncertainty estimates are often reported but rarely used to guide decisions. We study the missing step: how uncertainty maps are converted into actionable policies such as accepting, flagging, or deferring predictions. We formulate segmentation as a two-stage pipeline, estimation followed by decision, and show that optimizing uncertainty alone fails to capture most of the achievable safety gains. Using retinal vessel segmentation benchmarks (DRIVE, STARE, CHASE_DB1), we evaluate two uncertainty sources (Monte Carlo Dropout and Test-Time Augmentation) combined with three deferral strategies, and introduce a simple confidence-aware deferral rule that prioritizes uncertain and low-confidence predictions. Our results show that the best method and policy combination removes up to 80 percent of segmentation errors at only 25 percent pixel deferral, while achieving strong cross-dataset robustness. We further show that calibration improvements do not translate to better decision quality, highlighting a disconnect between standard uncertainty metrics and real-world utility. These findings suggest that uncertainty should be evaluated based on the decisions it enables, rather than in isolation.