ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection
This addresses a specific challenge in surgical robotics for improving registration accuracy in procedures like prostatectomy, but it is incremental as it builds on existing segmentation models.
The paper tackles the problem of accurately detecting surgical instrument tips in robot-assisted laparoscopic radical prostatectomy, proposing a deep learning method that uses part-level segmentation masks as input, which outperforms a hand-crafted image-processing approach on simulated and real datasets.
In robot-assisted laparoscopic radical prostatectomy (RALP), the location of the instrument tip is important to register the ultrasound frame with the laparoscopic camera frame. A long-standing limitation is that the instrument tip position obtained from the da Vinci API is inaccurate and requires hand-eye calibration. Thus, directly computing the position of the tool tip in the camera frame using the vision-based method becomes an attractive solution. Besides, surgical instrument tip detection is the key component of other tasks, like surgical skill assessment and surgery automation. However, this task is challenging due to the small size of the tool tip and the articulation of the surgical instrument. Surgical instrument segmentation becomes relatively easy due to the emergence of the Segmentation Foundation Model, i.e., Segment Anything. Based on this advancement, we explore the deep learning-based surgical instrument tip detection approach that takes the part-level instrument segmentation mask as input. Comparison experiments with a hand-crafted image-processing approach demonstrate the superiority of the proposed method on simulated and real datasets.