Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps
This addresses depth completion in real-world dynamic scenes where color images are often blurry, improving accuracy for applications like robotics or AR, though it is incremental by extending existing methods to handle blur.
The paper tackles depth map completion from sparse, noisy measurements by jointly exploiting blurry color image sequences and sparse depth maps, achieving state-of-the-art performance in both outdoor and indoor scenarios.
We aim at predicting a complete and high-resolution depth map from incomplete, sparse and noisy depth measurements. Existing methods handle this problem either by exploiting various regularizations on the depth maps directly or resorting to learning based methods. When the corresponding color images are available, the correlation between the depth maps and the color images are used to improve the completion performance, assuming the color images are clean and sharp. However, in real world dynamic scenes, color images are often blurry due to the camera motion and the moving objects in the scene. In this paper, we propose to tackle the problem of depth map completion by jointly exploiting the blurry color image sequences and the sparse depth map measurements, and present an energy minimization based formulation to simultaneously complete the depth maps, estimate the scene flow and deblur the color images. Our experimental evaluations on both outdoor and indoor scenarios demonstrate the state-of-the-art performance of our approach.