Single-Shot Metric Depth from Focused Plenoptic Cameras
This addresses the problem of metric depth estimation for robotics and computer vision applications, offering a compact alternative to traditional methods, though it appears incremental by combining existing techniques with a new dataset.
The paper tackles metric depth estimation from visual sensors by proposing a pipeline that predicts dense metric depth from a single plenoptic camera shot, using machine learning to generate a sparse metric point cloud and scaling a dense relative depth map, and validates it with a new Light Field & Stereo Image Dataset (LFS), showing accurate predictions.
Metric depth estimation from visual sensors is crucial for robots to perceive, navigate, and interact with their environment. Traditional range imaging setups, such as stereo or structured light cameras, face hassles including calibration, occlusions, and hardware demands, with accuracy limited by the baseline between cameras. Single- and multi-view monocular depth offers a more compact alternative, but is constrained by the unobservability of the metric scale. Light field imaging provides a promising solution for estimating metric depth by using a unique lens configuration through a single device. However, its application to single-view dense metric depth is under-addressed mainly due to the technology's high cost, the lack of public benchmarks, and proprietary geometrical models and software. Our work explores the potential of focused plenoptic cameras for dense metric depth. We propose a novel pipeline that predicts metric depth from a single plenoptic camera shot by first generating a sparse metric point cloud using machine learning, which is then used to scale and align a dense relative depth map regressed by a foundation depth model, resulting in dense metric depth. To validate it, we curated the Light Field & Stereo Image Dataset (LFS) of real-world light field images with stereo depth labels, filling a current gap in existing resources. Experimental results show that our pipeline produces accurate metric depth predictions, laying a solid groundwork for future research in this field.