ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images
This addresses the need for more accurate 3D reconstruction in applications like city modeling, though it is incremental as it builds on existing stereo matching pipelines.
The paper tackles the problem of noisy and inaccurate digital surface models (DSMs) from high-resolution satellite stereo images by introducing ResDepth, a convolutional neural network that learns a geometric prior to refine DSMs, resulting in consistent quantitative and qualitative improvements across experiments.
Modern optical satellite sensors enable high-resolution stereo reconstruction from space. But the challenging imaging conditions when observing the Earth from space push stereo matching to its limits. In practice, the resulting digital surface models (DSMs) are fairly noisy and often do not attain the accuracy needed for high-resolution applications such as 3D city modeling. Arguably, stereo correspondence based on low-level image similarity is insufficient and should be complemented with a-priori knowledge about the expected surface geometry beyond basic local smoothness. To that end, we introduce ResDepth, a convolutional neural network that learns such an expressive geometric prior from example data. ResDepth refines an initial, raw stereo DSM while conditioning the refinement on the images. I.e., it acts as a smart, learned post-processing filter and can seamlessly complement any stereo matching pipeline. In a series of experiments, we find that the proposed method consistently improves stereo DSMs both quantitatively and qualitatively. We show that the prior encoded in the network weights captures meaningful geometric characteristics of urban design, which also generalize across different districts and even from one city to another. Moreover, we demonstrate that, by training on a variety of stereo pairs, ResDepth can acquire a sufficient degree of invariance against variations in imaging conditions and acquisition geometry.