Temporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation
This work addresses surgical scene understanding for robot-assisted laparoscopic surgery, representing a domain-specific advancement with strong performance gains.
The paper tackles surgical scene segmentation in robot-assisted laparoscopic surgery by addressing limitations of static image approaches and dynamic video complexities, proposing a temporal asymmetric feature propagation network that achieves +16.4% mIoU on EndoVis2018 and +3.3% mAP on Endoscapes2023 compared to current SOTA methods.
Surgical scene segmentation is crucial for robot-assisted laparoscopic surgery understanding. Current approaches face two challenges: (i) static image limitations including ambiguous local feature similarities and fine-grained structural details, and (ii) dynamic video complexities arising from rapid instrument motion and persistent visual occlusions. While existing methods mainly focus on spatial feature extraction, they fundamentally overlook temporal dependencies in surgical video streams. To address this, we present temporal asymmetric feature propagation network, a bidirectional attention architecture enabling cross-frame feature propagation. The proposed method contains a temporal query propagator that integrates multi-directional consistency constraints to enhance frame-specific feature representation, and an aggregated asymmetric feature pyramid module that preserves discriminative features for anatomical structures and surgical instruments. Our framework uniquely enables both temporal guidance and contextual reasoning for surgical scene understanding. Comprehensive evaluations on two public benchmarks show the proposed method outperforms the current SOTA methods by a large margin, with +16.4\% mIoU on EndoVis2018 and +3.3\% mAP on Endoscapes2023. The code will be publicly available after paper acceptance.