Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
This addresses the problem of 3D reconstruction from indoor panoramas for applications like robotics or VR, but it is incremental as it builds on existing planar reconstruction methods with specific adaptations.
The paper tackles indoor panorama planar 3D reconstruction by approximating scenes with horizontal and vertical planes using a divide-and-conquer strategy and a yaw-invariant reparameterization for CNNs, outperforming baselines by a large margin on a new dataset.
Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity. We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes. To this end, we propose an effective divide-and-conquer strategy that divides pixels based on their plane orientation estimation; then, the succeeding instance segmentation module conquers the task of planes clustering more easily in each plane orientation group. Besides, parameters of V-planes depend on camera yaw rotation, but translation-invariant CNNs are less aware of the yaw change. We thus propose a yaw-invariant V-planar reparameterization for CNNs to learn. We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H\&V-planes (referred to as PanoH&V dataset) and adopt state-of-the-art planar reconstruction methods to predict H\&V-planes as our baselines. Our method outperforms the baselines by a large margin on the proposed dataset.