OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas
This work improves 3D layout estimation for indoor scenes, benefiting applications in robotics and augmented reality, but it is incremental as it builds on existing methods with a specific technical enhancement.
The paper tackles the problem of 3D room layout reconstruction from single RGB panoramas by addressing distortions in panoramic images that degrade performance with standard convolutions, and it shows that using spherical convolutions reduces error in distorted regions by about 25% and outperforms state-of-the-art by approximately 4% on benchmark datasets.
Given a single RGB panorama, the goal of 3D layout reconstruction is to estimate the room layout by predicting the corners, floor boundary, and ceiling boundary. A common approach has been to use standard convolutional networks to predict the corners and boundaries, followed by post-processing to generate the 3D layout. However, the space-varying distortions in panoramic images are not compatible with the translational equivariance property of standard convolutions, thus degrading performance. Instead, we propose to use spherical convolutions. The resulting network, which we call OmniLayout performs convolutions directly on the sphere surface, sampling according to inverse equirectangular projection and hence invariant to equirectangular distortions. Using a new evaluation metric, we show that our network reduces the error in the heavily distorted regions (near the poles) by approx 25 % when compared to standard convolutional networks. Experimental results show that OmniLayout outperforms the state-of-the-art by approx 4% on two different benchmark datasets (PanoContext and Stanford 2D-3D). Code is available at https://github.com/rshivansh/OmniLayout.