Symmetric Dilated Convolution for Surgical Gesture Recognition
This work addresses the problem of automatic surgical gesture recognition for intra-operative assistance and skill assessment, offering an incremental improvement over prior methods.
The paper tackles surgical gesture recognition from RGB videos by proposing a symmetric dilated convolutional architecture with self-attention, achieving state-of-the-art results with improvements of ~6 points in frame-wise accuracy and F1@50 score on the JIGSAWS dataset.
Automatic surgical gesture recognition is a prerequisite of intra-operative computer assistance and objective surgical skill assessment. Prior works either require additional sensors to collect kinematics data or have limitations on capturing temporal information from long and untrimmed surgical videos. To tackle these challenges, we propose a novel temporal convolutional architecture to automatically detect and segment surgical gestures with corresponding boundaries only using RGB videos. We devise our method with a symmetric dilation structure bridged by a self-attention module to encode and decode the long-term temporal patterns and establish the frame-to-frame relationship accordingly. We validate the effectiveness of our approach on a fundamental robotic suturing task from the JIGSAWS dataset. The experiment results demonstrate the ability of our method on capturing long-term frame dependencies, which largely outperform the state-of-the-art methods on the frame-wise accuracy up to ~6 points and the F1@50 score ~6 points.