Monocular Spherical Depth Estimation with Explicitly Connected Weak Layout Cues
This work addresses the problem of indoor geometric perception for researchers and applications in 3D scanning, though it is incremental as it builds on existing tasks with a new dataset and coupling method.
The authors tackled the lack of datasets with both layout annotations and dense depth maps for indoor scenes by generating a 360V dataset with multi-view stereo data and automatically generated weak layout cues, and they developed a model that explicitly couples depth estimation and layout reconstruction, achieving increased performance in both tasks.
Spherical cameras capture scenes in a holistic manner and have been used for room layout estimation. Recently, with the availability of appropriate datasets, there has also been progress in depth estimation from a single omnidirectional image. While these two tasks are complementary, few works have been able to explore them in parallel to advance indoor geometric perception, and those that have done so either relied on synthetic data, or used small scale datasets, as few options are available that include both layout annotations and dense depth maps in real scenes. This is partly due to the necessity of manual annotations for room layouts. In this work, we move beyond this limitation and generate a 360 geometric vision (360V) dataset that includes multiple modalities, multi-view stereo data and automatically generated weak layout cues. We also explore an explicit coupling between the two tasks to integrate them into a singleshot trained model. We rely on depth-based layout reconstruction and layout-based depth attention, demonstrating increased performance across both tasks. By using single 360 cameras to scan rooms, the opportunity for facile and quick building-scale 3D scanning arises.