Learning to Reconstruct 3D Manhattan Wireframes from a Single Image
This addresses the challenge of creating compact 3D representations from images for high-level vision tasks, though it appears incremental as it builds on existing learning-based wireframe detection methods.
The paper tackles the problem of reconstructing 3D Manhattan wireframes from a single image by exploiting global structural regularities, resulting in a simpler and more unified network that improves 2D wireframe detection and enables full 3D reconstruction suitable for applications like AR and CAD.
In this paper, we propose a method to obtain a compact and accurate 3D wireframe representation from a single image by effectively exploiting global structural regularities. Our method trains a convolutional neural network to simultaneously detect salient junctions and straight lines, as well as predict their 3D depth and vanishing points. Compared with the state-of-the-art learning-based wireframe detection methods, our network is simpler and more unified, leading to better 2D wireframe detection. With global structural priors from parallelism, our method further reconstructs a full 3D wireframe model, a compact vector representation suitable for a variety of high-level vision tasks such as AR and CAD. We conduct extensive evaluations on a large synthetic dataset of urban scenes as well as real images. Our code and datasets have been made public at https://github.com/zhou13/shapeunity.