Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organs Towards Robot-assisted Intubation
This work addresses a domain-specific challenge in medical robotics for oropharyngeal organ segmentation, with incremental improvements in adapting simulated data to real-world applications.
The paper tackles the problem of limited real endoscopic images for training deep-learning models in robot-assisted tracheal intubation by generating a virtual dataset using the SOFA framework and proposing a domain adaptive Sim-to-Real method with IoU-Ranking Blend and style-transfer techniques, resulting in improved segmentation accuracy and training stability.
Robotic-assisted tracheal intubation requires the robot to distinguish anatomical features like an experienced physician using deep-learning techniques. However, real datasets of oropharyngeal organs are limited due to patient privacy issues, making it challenging to train deep-learning models for accurate image segmentation. We hereby consider generating a new data modality through a virtual environment to assist the training process. Specifically, this work introduces a virtual dataset generated by the Simulation Open Framework Architecture (SOFA) framework to overcome the limited availability of actual endoscopic images. We also propose a domain adaptive Sim-to-Real method for oropharyngeal organ image segmentation, which employs an image blending strategy called IoU-Ranking Blend (IRB) and style-transfer techniques to address discrepancies between datasets. Experimental results demonstrate the superior performance of the proposed approach with domain adaptive models, improving segmentation accuracy and training stability. In the practical application, the trained segmentation model holds great promise for robot-assisted intubation surgery and intelligent surgical navigation.