MVSAnywhere: Zero-Shot Multi-View Stereo
It addresses the problem of domain generalization in depth estimation for computer vision applications, representing an incremental improvement over existing methods.
The paper tackled the challenge of generalizing multi-view stereo depth estimation across diverse domains and depth ranges, achieving state-of-the-art zero-shot performance on the Robust Multi-View Depth Benchmark.
Computing accurate depth from multiple views is a fundamental and longstanding challenge in computer vision. However, most existing approaches do not generalize well across different domains and scene types (e.g. indoor vs. outdoor). Training a general-purpose multi-view stereo model is challenging and raises several questions, e.g. how to best make use of transformer-based architectures, how to incorporate additional metadata when there is a variable number of input views, and how to estimate the range of valid depths which can vary considerably across different scenes and is typically not known a priori? To address these issues, we introduce MVSA, a novel and versatile Multi-View Stereo architecture that aims to work Anywhere by generalizing across diverse domains and depth ranges. MVSA combines monocular and multi-view cues with an adaptive cost volume to deal with scale-related issues. We demonstrate state-of-the-art zero-shot depth estimation on the Robust Multi-View Depth Benchmark, surpassing existing multi-view stereo and monocular baselines.