Bimodal Stereo: Joint Shape and Pose Estimation from Color-Depth Image Pair
This addresses mutual calibration challenges in multi-modal data registration for computer vision applications, but appears incremental as it builds on existing Shape-from-Shading and registration techniques.
The paper tackles the 'Bimodal Stereo' problem of estimating camera pose from an uncalibrated color image and depth map pair by proposing an iterative Shape-from-Shading framework that simultaneously refines shape and pose, with experiments showing desirable refinement and recovery of depth, shape, and pose.
Mutual calibration between color and depth cameras is a challenging topic in multi-modal data registration. In this paper, we are confronted with a "Bimodal Stereo" problem, which aims to solve camera pose from a pair of an uncalibrated color image and a depth map from different views automatically. To address this problem, an iterative Shape-from-Shading (SfS) based framework is proposed to estimate shape and pose simultaneously. In the pipeline, the estimated shape is refined by the shape prior from the given depth map under the estimated pose. Meanwhile, the estimated pose is improved by the registration of estimated shape and shape from given depth map. We also introduce a shading based refinement in the pipeline to address noisy depth map with holes. Extensive experiments showed that through our method, both the depth map, the recovered shape as well as its pose can be desirably refined and recovered.