Dense image registration and deformable surface reconstruction in presence of occlusions and minimal texture
This addresses a challenging problem in computer vision for applications like medical imaging or robotics where surfaces lack texture and are partially hidden.
The paper tackles 3D reconstruction of poorly textured, occluded surfaces from monocular images, achieving state-of-the-art results on both well and poorly textured surfaces.
Deformable surface tracking from monocular images is well-known to be under-constrained. Occlusions often make the task even more challenging, and can result in failure if the surface is not sufficiently textured. In this work, we explicitly address the problem of 3D reconstruction of poorly textured, occluded surfaces, proposing a framework based on a template-matching approach that scales dense robust features by a relevancy score. Our approach is extensively compared to current methods employing both local feature matching and dense template alignment. We test on standard datasets as well as on a new dataset (that will be made publicly available) of a sparsely textured, occluded surface. Our framework achieves state-of-the-art results for both well and poorly textured, occluded surfaces.