Depth Completion Using a View-constrained Deep Prior
This work addresses depth completion for 3D reconstruction in computer vision, but it is incremental as it adapts an existing prior to a new domain.
The paper tackled the problem of completing and refining noisy, incomplete depth maps by extending the deep image prior (DIP) concept to depth images, using a view-constrained photo-consistency loss from nearby camera viewpoints. The result showed that the refined depth maps were more accurate and complete, leading to higher-quality dense 3D models after fusion.
Recent work has shown that the structure of convolutional neural networks (CNNs) induces a strong prior that favors natural images. This prior, known as a deep image prior (DIP), is an effective regularizer in inverse problems such as image denoising and inpainting. We extend the concept of the DIP to depth images. Given color images and noisy and incomplete target depth maps, we optimize a randomly-initialized CNN model to reconstruct a depth map restored by virtue of using the CNN network structure as a prior combined with a view-constrained photo-consistency loss. This loss is computed using images from a geometrically calibrated camera from nearby viewpoints. We apply this deep depth prior for inpainting and refining incomplete and noisy depth maps within both binocular and multi-view stereo pipelines. Our quantitative and qualitative evaluation shows that our refined depth maps are more accurate and complete, and after fusion, produces dense 3D models of higher quality.