Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification
This work addresses over-segmentation issues in surgical skill assessment and training, offering a novel approach for domain-specific applications.
The paper tackles surgical gesture segmentation and classification by modeling it as a sequential decision-making process using deep reinforcement learning, resulting in improved edit scores compared to state-of-the-art methods on the JIGSAWS dataset.
Recognition of surgical gesture is crucial for surgical skill assessment and efficient surgery training. Prior works on this task are based on either variant graphical models such as HMMs and CRFs, or deep learning models such as Recurrent Neural Networks and Temporal Convolutional Networks. Most of the current approaches usually suffer from over-segmentation and therefore low segment-level edit scores. In contrast, we present an essentially different methodology by modeling the task as a sequential decision-making process. An intelligent agent is trained using reinforcement learning with hierarchical features from a deep model. Temporal consistency is integrated into our action design and reward mechanism to reduce over-segmentation errors. Experiments on JIGSAWS dataset demonstrate that the proposed method performs better than state-of-the-art methods in terms of the edit score and on par in frame-wise accuracy. Our code will be released later.