Learning to Generate 3D Representations of Building Roofs Using Single-View Aerial Imagery
This work addresses the challenge of 3D reconstruction for building roofs in remote sensing, enabling efficient modeling from limited data, though it is incremental as it applies an existing method to a new domain.
The paper tackles the problem of generating 3D roof meshes from single-view aerial images by learning conditional distributions based on regular patterns, achieving robust performance even at lower resolutions and producing realistic representations for out-of-distribution samples.
We present a novel pipeline for learning the conditional distribution of a building roof mesh given pixels from an aerial image, under the assumption that roof geometry follows a set of regular patterns. Unlike alternative methods that require multiple images of the same object, our approach enables estimating 3D roof meshes using only a single image for predictions. The approach employs the PolyGen, a deep generative transformer architecture for 3D meshes. We apply this model in a new domain and investigate the sensitivity of the image resolution. We propose a novel metric to evaluate the performance of the inferred meshes, and our results show that the model is robust even at lower resolutions, while qualitatively producing realistic representations for out-of-distribution samples.