Surgical Gesture Recognition with Optical Flow only
This work addresses the problem of surgical gesture recognition for medical applications by providing a video-based method that avoids the need for additional recording devices, though it is incremental as it adapts an existing optical flow ConvNet approach.
The paper tackles surgical gesture recognition by using only dense optical flow from video data as an alternative to kinematic data, achieving competitive results on the JIGSAWS dataset with more robust performance and less standard deviation.
In this paper, we address the open research problem of surgical gesture recognition using motion cues from video data only. We adapt Optical flow ConvNets initially proposed by Simonyan et al.. While Simonyan uses both RGB frames and dense optical flow, we use only dense optical flow representations as input to emphasize the role of motion in surgical gesture recognition, and present it as a robust alternative to kinematic data. We also overcome one of the limitations of Optical flow ConvNets by initializing our model with cross modality pre-training. A large number of promising studies that address surgical gesture recognition highly rely on kinematic data which requires additional recording devices. To our knowledge, this is the first paper that addresses surgical gesture recognition using dense optical flow information only. We achieve competitive results on JIGSAWS dataset, moreover, our model achieves more robust results with less standard deviation, which suggests optical flow information can be used as an alternative to kinematic data for the recognition of surgical gestures.