A Surface-normal Based Neural Framework for Colonoscopy Reconstruction
This work addresses colonoscopy reconstruction for medical imaging, offering incremental improvements in accuracy for domain-specific applications.
The paper tackles the challenge of 3D surface reconstruction from colonoscopy video by developing a two-step neural framework based on surface normal vectors, which significantly improves reconstruction quality, achieving low depth error that reduces the need for post-processing for clinical use.
Reconstructing a 3D surface from colonoscopy video is challenging due to illumination and reflectivity variation in the video frame that can cause defective shape predictions. Aiming to overcome this challenge, we utilize the characteristics of surface normal vectors and develop a two-step neural framework that significantly improves the colonoscopy reconstruction quality. The normal-based depth initialization network trained with self-supervised normal consistency loss provides depth map initialization to the normal-depth refinement module, which utilizes the relationship between illumination and surface normals to refine the frame-wise normal and depth predictions recursively. Our framework's depth accuracy performance on phantom colonoscopy data demonstrates the value of exploiting the surface normals in colonoscopy reconstruction, especially on en face views. Due to its low depth error, the prediction result from our framework will require limited post-processing to be clinically applicable for real-time colonoscopy reconstruction.