ColonMapper: topological mapping and localization for colonoscopy
This addresses the challenge of reliable navigation and relocalization in medical colonoscopy procedures, which is incremental by combining existing methods like transformers-based matching with a new deep global descriptor and Bayesian filter.
The paper tackles the problem of mapping and localization in colonoscopy videos under significant shape and illumination changes, achieving autonomous map building and localization within the same or different colonoscopies of the same patient.
We propose a topological mapping and localization system able to operate on real human colonoscopies, despite significant shape and illumination changes. The map is a graph where each node codes a colon location by a set of real images, while edges represent traversability between nodes. For close-in-time images, where scene changes are minor, place recognition can be successfully managed with the recent transformers-based local feature matching algorithms. However, under long-term changes -- such as different colonoscopies of the same patient -- feature-based matching fails. To address this, we train on real colonoscopies a deep global descriptor achieving high recall with significant changes in the scene. The addition of a Bayesian filter boosts the accuracy of long-term place recognition, enabling relocalization in a previously built map. Our experiments show that ColonMapper is able to autonomously build a map and localize against it in two important use cases: localization within the same colonoscopy or within different colonoscopies of the same patient. Code: https://github.com/jmorlana/ColonMapper.