PanelNet: Understanding 360 Indoor Environment via Panel Representation
This work addresses the challenge of interpreting 360-degree indoor images for applications like robotics or virtual reality, representing an incremental improvement with a new method for handling panoramic distortion.
The paper tackles the problem of understanding indoor environments from 360-degree panoramas by introducing PanelNet, a framework that uses a novel panel representation to improve depth estimation, layout estimation, and semantic segmentation, achieving superior performance in depth estimation and competitive results in other tasks.
Indoor 360 panoramas have two essential properties. (1) The panoramas are continuous and seamless in the horizontal direction. (2) Gravity plays an important role in indoor environment design. By leveraging these properties, we present PanelNet, a framework that understands indoor environments using a novel panel representation of 360 images. We represent an equirectangular projection (ERP) as consecutive vertical panels with corresponding 3D panel geometry. To reduce the negative impact of panoramic distortion, we incorporate a panel geometry embedding network that encodes both the local and global geometric features of a panel. To capture the geometric context in room design, we introduce Local2Global Transformer, which aggregates local information within a panel and panel-wise global context. It greatly improves the model performance with low training overhead. Our method outperforms existing methods on indoor 360 depth estimation and shows competitive results against state-of-the-art approaches on the task of indoor layout estimation and semantic segmentation.