Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction
This work addresses surgical data science and computer-aided intervention by enhancing gesture recognition, but it is incremental as it builds on existing methods with a novel multi-task formulation.
The paper tackles the problem of surgical gesture recognition from robotic kinematic data, which is challenging due to variability in surgical demonstrations, and proposes a multi-task recurrent neural network that simultaneously recognizes gestures and estimates surgical task progress, showing improved recognition performance on the JIGSAWS dataset without extra labeling or training.
Surgical gesture recognition is important for surgical data science and computer-aided intervention. Even with robotic kinematic information, automatically segmenting surgical steps presents numerous challenges because surgical demonstrations are characterized by high variability in style, duration and order of actions. In order to extract discriminative features from the kinematic signals and boost recognition accuracy, we propose a multi-task recurrent neural network for simultaneous recognition of surgical gestures and estimation of a novel formulation of surgical task progress. To show the effectiveness of the presented approach, we evaluate its application on the JIGSAWS dataset, that is currently the only publicly available dataset for surgical gesture recognition featuring robot kinematic data. We demonstrate that recognition performance improves in multi-task frameworks with progress estimation without any additional manual labelling and training.