XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface
This enables automated extraction of camera metadata to support downstream tasks like tool tracking and skill assessment in robot-assisted surgery, but it is incremental as it applies an existing method to a new domain-specific problem.
The researchers tackled the problem of automatically detecting camera activation in the Da Vinci Xi surgical system's user interface, achieving F1-scores between 0.993 and 1.000 for binary detection and perfect localization across over 70,000 frames.
Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.