MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry
This addresses the limitation of supervised MVS methods for scenarios where labeled data is unavailable, enabling broader application in 3D reconstruction.
The paper tackles the problem of multi-view stereo (MVS) requiring dense depth map supervision by proposing the first unsupervised learning-based MVS network that predicts depth maps from multi-view images without ground-truth 3D data, achieving effective results and excellent generalization ability on multiple datasets.
The success of existing deep-learning based multi-view stereo (MVS) approaches greatly depends on the availability of large-scale supervision in the form of dense depth maps. Such supervision, while not always possible, tends to hinder the generalization ability of the learned models in never-seen-before scenarios. In this paper, we propose the first unsupervised learning based MVS network, which learns the multi-view depth maps from the input multi-view images and does not need ground-truth 3D training data. Our network is symmetric in predicting depth maps for all views simultaneously, where we enforce cross-view consistency of multi-view depth maps during both training and testing stages. Thus, the learned multi-view depth maps naturally comply with the underlying 3D scene geometry. Besides, our network also learns the multi-view occlusion maps, which further improves the robustness of our network in handling real-world occlusions. Experimental results on multiple benchmarking datasets demonstrate the effectiveness of our network and the excellent generalization ability.