Learning to Adapt Multi-View Stereo by Self-Supervision
This work addresses the challenge of robust generalization for multi-view stereo reconstruction across different environments, which is important for computer vision applications, but it appears incremental as it builds on existing self-supervised and meta-learning techniques.
The paper tackles the problem of 3D scene reconstruction from multiple views by proposing an adaptive learning approach using model-agnostic meta-learning (MAML) to train a deep neural network for improved adaptability to new domains through self-supervised training, resulting in effective learning of multi-view stereo reconstruction in new domains.
3D scene reconstruction from multiple views is an important classical problem in computer vision. Deep learning based approaches have recently demonstrated impressive reconstruction results. When training such models, self-supervised methods are favourable since they do not rely on ground truth data which would be needed for supervised training and is often difficult to obtain. Moreover, learned multi-view stereo reconstruction is prone to environment changes and should robustly generalise to different domains. We propose an adaptive learning approach for multi-view stereo which trains a deep neural network for improved adaptability to new target domains. We use model-agnostic meta-learning (MAML) to train base parameters which, in turn, are adapted for multi-view stereo on new domains through self-supervised training. Our evaluations demonstrate that the proposed adaptation method is effective in learning self-supervised multi-view stereo reconstruction in new domains.