Metrics Matter in Surgical Phase Recognition
This work addresses evaluation inconsistencies in surgical phase recognition, which is incremental but important for reliable benchmarking and clinical translation in computer-assisted surgery.
The paper identifies inconsistencies in evaluation protocols for surgical phase recognition methods on the Cholec80 benchmark, summarizing common deviations and providing a structured overview of previously reported results to enable more meaningful comparisons.
Surgical phase recognition is a basic component for different context-aware applications in computer- and robot-assisted surgery. In recent years, several methods for automatic surgical phase recognition have been proposed, showing promising results. However, a meaningful comparison of these methods is difficult due to differences in the evaluation process and incomplete reporting of evaluation details. In particular, the details of metric computation can vary widely between different studies. To raise awareness of potential inconsistencies, this paper summarizes common deviations in the evaluation of phase recognition algorithms on the Cholec80 benchmark. In addition, a structured overview of previously reported evaluation results on Cholec80 is provided, taking known differences in evaluation protocols into account. Greater attention to evaluation details could help achieve more consistent and comparable results on the surgical phase recognition task, leading to more reliable conclusions about advancements in the field and, finally, translation into clinical practice.