Data Driven Robust Image Guided Depth Map Restoration
This work addresses depth map restoration for applications in computer vision and robotics, but it is incremental as it builds on existing energy optimization frameworks with novel robust terms and adaptive parameters.
The paper tackles the problem of restoring degraded depth maps from cameras like Kinect and ToF, which suffer from missing data, noise, and low resolution, by proposing a robust method that uses color image guidance and achieves high-quality results with sharp discontinuities and reduced artifacts, outperforming state-of-the-art methods in experiments.
Depth maps captured by modern depth cameras such as Kinect and Time-of-Flight (ToF) are usually contaminated by missing data, noises and suffer from being of low resolution. In this paper, we present a robust method for high-quality restoration of a degraded depth map with the guidance of the corresponding color image. We solve the problem in an energy optimization framework that consists of a novel robust data term and smoothness term. To accommodate not only the noise but also the inconsistency between depth discontinuities and the color edges, we model both the data term and smoothness term with a robust exponential error norm function. We propose to use Iteratively Re-weighted Least Squares (IRLS) methods for efficiently solving the resulting highly non-convex optimization problem. More importantly, we further develop a data-driven adaptive parameter selection scheme to properly determine the parameter in the model. We show that the proposed approach can preserve fine details and sharp depth discontinuities even for a large upsampling factor ($8\times$ for example). Experimental results on both simulated and real datasets demonstrate that the proposed method outperforms recent state-of-the-art methods in coping with the heavy noise, preserving sharp depth discontinuities and suppressing the texture copy artifacts.