From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction
This addresses the issue of improving 3D-aware methods for MDP, which is crucial for applications like 3D scene completion, though it is incremental as it focuses on evaluation rather than a new prediction method.
The paper tackles the problem of benchmark over-fitting and inadequate metrics in monocular depth prediction (MDP), which limit the development of methods that accurately estimate 3D scene structure, by proposing a new set of 3D geometry evaluation metrics and a novel indoor benchmark called RIO-D3D, based on real-world RGB-D reconstructions.
There have been numerous recently proposed methods for monocular depth prediction (MDP) coupled with the equally rapid evolution of benchmarking tools. However, we argue that MDP is currently witnessing benchmark over-fitting and relying on metrics that are only partially helpful to gauge the usefulness of the predictions for 3D applications. This limits the design and development of novel methods that are truly aware of - and improving towards estimating - the 3D structure of the scene rather than optimizing 2D-based distances. In this work, we aim to bring structural awareness to MDP, an inherently 3D task, by exhibiting the limits of evaluation metrics towards assessing the quality of the 3D geometry. We propose a set of metrics well suited to evaluate the 3D geometry of MDP approaches and a novel indoor benchmark, RIO-D3D, crucial for the proposed evaluation methodology. Our benchmark is based on a real-world dataset featuring high-quality rendered depth maps obtained from RGB-D reconstructions. We further demonstrate this to help benchmark the closely-tied task of 3D scene completion.