An Adaptive Framework for Learning Unsupervised Depth Completion
This is an incremental improvement for computer vision researchers working on depth completion tasks.
The paper tackles the problem of inferring dense depth maps from color images and sparse depth measurements by developing an adaptive weighting scheme that unifies regularization and co-visibility estimation. The method improves performance of existing unsupervised depth completion approaches on public benchmarks without adding trainable parameters or increasing inference time.
We present a method to infer a dense depth map from a color image and associated sparse depth measurements. Our main contribution lies in the design of an annealing process for determining co-visibility (occlusions, disocclusions) and the degree of regularization to impose on the model. We show that regularization and co-visibility are related via the fitness (residual) of model to data and both can be unified into a single framework to improve the learning process. Our method is an adaptive weighting scheme that guides optimization by measuring the residual at each pixel location over each training step for (i) estimating a soft visibility mask and (ii) determining the amount of regularization. We demonstrate the effectiveness our method by applying it to several recent unsupervised depth completion methods and improving their performance on public benchmark datasets, without incurring additional trainable parameters or increase in inference time. Code available at: https://github.com/alexklwong/adaframe-depth-completion.