Tracking Everything in Robotic-Assisted Surgery
This work is significant for researchers and developers in robotic surgery, as it provides a new dataset and a more robust tracking method to improve scene comprehension and robot autonomy in RAMIS, addressing limitations of existing methods in complex surgical scenarios.
This paper addresses the challenge of accurate and dense long-term tracking of tissues and instruments in Robotic-Assisted Minimally Invasive Surgery (RAMIS) videos. The authors introduce a new annotated surgical tracking dataset and propose SurgMotion, a novel tracking method that outperforms most TAP-based algorithms, especially in challenging medical videos.
Accurate tracking of tissues and instruments in videos is crucial for Robotic-Assisted Minimally Invasive Surgery (RAMIS), as it enables the robot to comprehend the surgical scene with precise locations and interactions of tissues and tools. Traditional keypoint-based sparse tracking is limited by featured points, while flow-based dense two-view matching suffers from long-term drifts. Recently, the Tracking Any Point (TAP) algorithm was proposed to overcome these limitations and achieve dense accurate long-term tracking. However, its efficacy in surgical scenarios remains untested, largely due to the lack of a comprehensive surgical tracking dataset for evaluation. To address this gap, we introduce a new annotated surgical tracking dataset for benchmarking tracking methods for surgical scenarios, comprising real-world surgical videos with complex tissue and instrument motions. We extensively evaluate state-of-the-art (SOTA) TAP-based algorithms on this dataset and reveal their limitations in challenging surgical scenarios, including fast instrument motion, severe occlusions, and motion blur, etc. Furthermore, we propose a new tracking method, namely SurgMotion, to solve the challenges and further improve the tracking performance. Our proposed method outperforms most TAP-based algorithms in surgical instruments tracking, and especially demonstrates significant improvements over baselines in challenging medical videos. Our code and dataset are available at https://github.com/zhanbh1019/SurgicalMotion.