A Deep Primal-Dual Network for Guided Depth Super-Resolution
This addresses the need for high-quality depth reconstruction in applications like robotics or augmented reality, but it is incremental as it builds on existing variational and deep learning techniques.
The paper tackles the problem of increasing spatial resolution of noisy, low-resolution depth images by combining a deep fully convolutional network with a non-local variational method in a deep primal-dual network, using a high-resolution intensity image as guidance, and shows that it outperforms state-of-the-art methods on multiple benchmarks.
In this paper we present a novel method to increase the spatial resolution of depth images. We combine a deep fully convolutional network with a non-local variational method in a deep primal-dual network. The joint network computes a noise-free, high-resolution estimate from a noisy, low-resolution input depth map. Additionally, a high-resolution intensity image is used to guide the reconstruction in the network. By unrolling the optimization steps of a first-order primal-dual algorithm and formulating it as a network, we can train our joint method end-to-end. This not only enables us to learn the weights of the fully convolutional network, but also to optimize all parameters of the variational method and its optimization procedure. The training of such a deep network requires a large dataset for supervision. Therefore, we generate high-quality depth maps and corresponding color images with a physically based renderer. In an exhaustive evaluation we show that our method outperforms the state-of-the-art on multiple benchmarks.