Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network
This addresses indoor scene reconstruction for applications like VR/AR, though it is incremental with novel loss functions and priors.
The paper tackles indoor 3D reconstruction from single 360° images by training a plane-aware CNN for depth, surface normal, and plane boundary estimation, outperforming existing methods in depth estimation and establishing a benchmark for surface normal prediction.
In this work we present a method to train a plane-aware convolutional neural network for dense depth and surface normal estimation as well as plane boundaries from a single indoor $360^\circ$ image. Using our proposed loss function, our network outperforms existing methods for single-view, indoor, omnidirectional depth estimation and provides an initial benchmark for surface normal prediction from $360^\circ$ images. Our improvements are due to the use of a novel plane-aware loss that leverages principal curvature as an indicator of planar boundaries. We also show that including geodesic coordinate maps as network priors provides a significant boost in surface normal prediction accuracy. Finally, we demonstrate how we can combine our network's outputs to generate high quality 3D "pop-up" models of indoor scenes.