Deep PatchMatch MVS with Learned Patch Coplanarity, Geometric Consistency and Adaptive Pixel Sampling
This work addresses 3D reconstruction challenges in computer vision, particularly for large-scale scenes with limited views, representing an incremental improvement over existing learning-based methods.
The paper tackled the problem of multi-view stereo (MVS) for large scenes with sparse views by improving learning-based methods with learned patch coplanarity, geometric consistency, and adaptive pixel sampling, resulting in 6-15% gains in accuracy and completeness on the ETH3D benchmark and outperforming non-learning state-of-the-art approaches.
Recent work in multi-view stereo (MVS) combines learnable photometric scores and regularization with PatchMatch-based optimization to achieve robust pixelwise estimates of depth, normals, and visibility. However, non-learning based methods still outperform for large scenes with sparse views, in part due to use of geometric consistency constraints and ability to optimize over many views at high resolution. In this paper, we build on learning-based approaches to improve photometric scores by learning patch coplanarity and encourage geometric consistency by learning a scaled photometric cost that can be combined with reprojection error. We also propose an adaptive pixel sampling strategy for candidate propagation that reduces memory to enable training on larger resolution with more views and a larger encoder. These modifications lead to 6-15% gains in accuracy and completeness on the challenging ETH3D benchmark, resulting in higher F1 performance than the widely used state-of-the-art non-learning approaches ACMM and ACMP.