Depth Field Networks for Generalizable Multi-view Scene Representation
This work addresses the need for more generalizable multi-view scene representations in 3D computer vision, reducing task-specific tuning and enhancing performance across domains.
The paper tackles the problem of poor domain generalization in 3D computer vision by proposing Depth Field Networks (DeFiNe), which learn an implicit, multi-view consistent scene representation without explicit geometric constraints, achieving state-of-the-art results in stereo and video depth estimation and improving zero-shot domain generalization by a wide margin.
Modern 3D computer vision leverages learning to boost geometric reasoning, mapping image data to classical structures such as cost volumes or epipolar constraints to improve matching. These architectures are specialized according to the particular problem, and thus require significant task-specific tuning, often leading to poor domain generalization performance. Recently, generalist Transformer architectures have achieved impressive results in tasks such as optical flow and depth estimation by encoding geometric priors as inputs rather than as enforced constraints. In this paper, we extend this idea and propose to learn an implicit, multi-view consistent scene representation, introducing a series of 3D data augmentation techniques as a geometric inductive prior to increase view diversity. We also show that introducing view synthesis as an auxiliary task further improves depth estimation. Our Depth Field Networks (DeFiNe) achieve state-of-the-art results in stereo and video depth estimation without explicit geometric constraints, and improve on zero-shot domain generalization by a wide margin.