Self-supervised 360$^{\circ}$ Room Layout Estimation
This work addresses the challenge of data scarcity in room layout estimation for applications like real estate virtual tours, though it is incremental as it adapts self-supervised techniques to a new domain.
The authors tackled the problem of panoramic room layout estimation without labeled data by proposing a self-supervised method using Differentiable Layout View Rendering and regularization losses, achieving the first self-supervised results on ZilloIndoor and MatterportLayout datasets.
We present the first self-supervised method to train panoramic room layout estimation models without any labeled data. Unlike per-pixel dense depth that provides abundant correspondence constraints, layout representation is sparse and topological, hindering the use of self-supervised reprojection consistency on images. To address this issue, we propose Differentiable Layout View Rendering, which can warp a source image to the target camera pose given the estimated layout from the target image. As each rendered pixel is differentiable with respect to the estimated layout, we can now train the layout estimation model by minimizing reprojection loss. Besides, we introduce regularization losses to encourage Manhattan alignment, ceiling-floor alignment, cycle consistency, and layout stretch consistency, which further improve our predictions. Finally, we present the first self-supervised results on ZilloIndoor and MatterportLayout datasets. Our approach also shows promising solutions in data-scarce scenarios and active learning, which would have an immediate value in the real estate virtual tour software. Code is available at https://github.com/joshua049/Stereo-360-Layout.