Structure-From-Motion and RGBD Depth Fusion
This addresses depth sensing gaps for applications such as robotic localization and mapping, but it is incremental as it combines existing technologies.
The paper tackles the problem of depth sensing limitations in RGBD sensors by fusing sensor depth measurements with Structure-from-Motion (SfM) depth estimates, resulting in an improved depth stream usable in contexts like distant surfaces (>5m), dark surfaces, and brightly lit scenes.
This article describes a technique to augment a typical RGBD sensor by integrating depth estimates obtained via Structure-from-Motion (SfM) with sensor depth measurements. Limitations in the RGBD depth sensing technology prevent capturing depth measurements in four important contexts: (1) distant surfaces (>5m), (2) dark surfaces, (3) brightly lit indoor scenes and (4) sunlit outdoor scenes. SfM technology computes depth via multi-view reconstruction from the RGB image sequence alone. As such, SfM depth estimates do not suffer the same limitations and may be computed in all four of the previously listed circumstances. This work describes a novel fusion of RGBD depth data and SfM-estimated depths to generate an improved depth stream that may be processed by one of many important downstream applications such as robotic localization and mapping, as well as object recognition and tracking.