SLAM Endoscopy enhanced by adversarial depth prediction
This addresses the problem of limited depth information in endoscopic procedures for medical professionals, though it appears incremental as it builds on existing SLAM and depth estimation methods.
The paper tackled the challenge of sparse features and lack of depth sensing in medical endoscopy SLAM by incorporating adversarially-trained CNN depth predictions from monocular images, enabling dense reconstruction and mosaicing in gastrointestinal tracts.
Medical endoscopy remains a challenging application for simultaneous localization and mapping (SLAM) due to the sparsity of image features and size constraints that prevent direct depth-sensing. We present a SLAM approach that incorporates depth predictions made by an adversarially-trained convolutional neural network (CNN) applied to monocular endoscopy images. The depth network is trained with synthetic images of a simple colon model, and then fine-tuned with domain-randomized, photorealistic images rendered from computed tomography measurements of human colons. Each image is paired with an error-free depth map for supervised adversarial learning. Monocular RGB images are then fused with corresponding depth predictions, enabling dense reconstruction and mosaicing as an endoscope is advanced through the gastrointestinal tract. Our preliminary results demonstrate that incorporating monocular depth estimation into a SLAM architecture can enable dense reconstruction of endoscopic scenes.