Understanding Annotation Error Propagation and Learning an Adaptive Policy for Expert Intervention in Barrett's Video Segmentation
This work addresses the challenge of reducing expert annotation effort while maintaining accuracy in medical video segmentation, which is incremental by building on existing semi-automatic tools like SAM2.
The paper tackled the problem of annotation error propagation in endoscopic video segmentation for Barrett's esophagus dysplasia, proposing a cost-aware framework called Learning-to-Re-Prompt (L2RP) that learns when to seek expert input, resulting in improved temporal consistency and superior performance over baselines on a private dataset and the SUN-SEG benchmark.
Accurate annotation of endoscopic videos is essential yet time-consuming, particularly for challenging datasets such as dysplasia in Barrett's esophagus, where the affected regions are irregular and lack clear boundaries. Semi-automatic tools like Segment Anything Model 2 (SAM2) can ease this process by propagating annotations across frames, but small errors often accumulate and reduce accuracy, requiring expert review and correction. To address this, we systematically study how annotation errors propagate across different prompt types, namely masks, boxes, and points, and propose Learning-to-Re-Prompt (L2RP), a cost-aware framework that learns when and where to seek expert input. By tuning a human-cost parameter, our method balances annotation effort and segmentation accuracy. Experiments on a private Barrett's dysplasia dataset and the public SUN-SEG benchmark demonstrate improved temporal consistency and superior performance over baseline strategies.