Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization
This addresses the challenge of reliable tool tracking for surgical robots, which is incremental as it builds on rendering-based methods to overcome computational and convergence issues.
The paper tackled the problem of accurate and efficient surgical instrument tracking for Robot-Assisted Minimally Invasive Surgery by incorporating CMA-ES evolutionary optimization into a rendering-based pipeline, resulting in significant improvements in both accuracy and runtime compared to prior methods.
Accurate and efficient tracking of surgical instruments is fundamental for Robot-Assisted Minimally Invasive Surgery. Although vision-based robot pose estimation has enabled markerless calibration without tedious physical setups, reliable tool tracking for surgical robots still remains challenging due to partial visibility and specialized articulation design of surgical instruments. Previous works in the field are usually prone to unreliable feature detections under degraded visual quality and data scarcity, whereas rendering-based methods often struggle with computational costs and suboptimal convergence. In this work, we incorporate CMA-ES, an evolutionary optimization strategy, into a versatile tracking pipeline that jointly estimates surgical instrument pose and joint configurations. Using batch rendering to efficiently evaluate multiple pose candidates in parallel, the method significantly reduces inference time and improves convergence robustness. The proposed framework further generalizes to joint angle-free and bi-manual tracking settings, making it suitable for both vision feedback control and online surgery video calibration. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method significantly outperforms prior approaches in both accuracy and runtime.