Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generation
This work addresses the challenge of documenting complex surgical interactions for clinical tasks, representing an incremental advance in surgical report generation.
The paper tackles the problem of generating accurate surgical reports for robot-assisted surgery by explicitly modeling the interactive relations between instruments and tissues, achieving improvements of 7.48% in BLEU-1 and 5.43% in ROUGE over state-of-the-art methods.
For robot-assisted surgery, an accurate surgical report reflects clinical operations during surgery and helps document entry tasks, post-operative analysis and follow-up treatment. It is a challenging task due to many complex and diverse interactions between instruments and tissues in the surgical scene. Although existing surgical report generation methods based on deep learning have achieved large success, they often ignore the interactive relation between tissues and instrumental tools, thereby degrading the report generation performance. This paper presents a neural network to boost surgical report generation by explicitly exploring the interactive relation between tissues and surgical instruments. We validate the effectiveness of our method on a widely-used robotic surgery benchmark dataset, and experimental results show that our network can significantly outperform existing state-of-the-art surgical report generation methods (e.g., 7.48% and 5.43% higher for BLEU-1 and ROUGE).