Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery
This addresses domain adaptation for surgical report generation, which is important for documentation and training in robotic surgery, but it appears incremental as it builds on existing adversarial learning and calibration techniques.
The paper tackles the problem of generating surgical reports in robotic surgery, where deep learning models often perform poorly when applied to new target domains, and shows that their proposed method improves performance in source and target domains across unsupervised, zero-shot, one-shot, and few-shot learning settings.
Generating a surgical report in robot-assisted surgery, in the form of natural language expression of surgical scene understanding, can play a significant role in document entry tasks, surgical training, and post-operative analysis. Despite the state-of-the-art accuracy of the deep learning algorithm, the deployment performance often drops when applied to the Target Domain (TD) data. For this purpose, we develop a multi-layer transformer-based model with the gradient reversal adversarial learning to generate a caption for the multi-domain surgical images that can describe the semantic relationship between instruments and surgical Region of Interest (ROI). In the gradient reversal adversarial learning scheme, the gradient multiplies with a negative constant and updates adversarially in backward propagation, discriminating between the source and target domains and emerging domain-invariant features. We also investigate model calibration with label smoothing technique and the effect of a well-calibrated model for the penultimate layer's feature representation and Domain Adaptation (DA). We annotate two robotic surgery datasets of MICCAI robotic scene segmentation and Transoral Robotic Surgery (TORS) with the captions of procedures and empirically show that our proposed method improves the performance in both source and target domain surgical reports generation in the manners of unsupervised, zero-shot, one-shot, and few-shot learning.