Bimodal Camera Pose Prediction for Endoscopy
This work addresses camera pose prediction for endoscopy, specifically in colonoscopy, which is incremental as it builds on existing methods with a new dataset and bimodal approach.
The paper tackles the challenge of 3D structure estimation in endoscopic scenes by proposing SimCol, a synthetic dataset for camera pose estimation in colonoscopy, and a novel bimodal method that outperforms prior unimodal approaches, demonstrating generalization to real colonoscopy sequences.
Deducing the 3D structure of endoscopic scenes from images is exceedingly challenging. In addition to deformation and view-dependent lighting, tubular structures like the colon present problems stemming from their self-occluding and repetitive anatomical structure. In this paper, we propose SimCol, a synthetic dataset for camera pose estimation in colonoscopy, and a novel method that explicitly learns a bimodal distribution to predict the endoscope pose. Our dataset replicates real colonoscope motion and highlights the drawbacks of existing methods. We publish 18k RGB images from simulated colonoscopy with corresponding depth and camera poses and make our data generation environment in Unity publicly available. We evaluate different camera pose prediction methods and demonstrate that, when trained on our data, they generalize to real colonoscopy sequences, and our bimodal approach outperforms prior unimodal work.