RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings
This addresses the need for efficient and accurate building outline extraction in geographic and cartographic applications, offering a method that reduces computational overhead and vertex redundancy without post-processing, though it is incremental as it builds on existing query-based and attention mechanisms.
The paper tackles the problem of extracting polygonal building outlines from aerial or satellite imagery by proposing RoIPoly, a novel query-based approach that formulates vertices as queries with constrained attention and uses learnable logit embeddings for vertex classification, eliminating the need for post-processing. On the CrowdAI dataset, it outperforms existing methods on most MS-COCO metrics, especially for small buildings, and achieves competitive results in polygon quality and vertex redundancy, while on the Structured3D dataset, it achieves second-best performance on most metrics for 2D floorplan reconstruction.
Polygonal building outlines are crucial for geographic and cartographic applications. The existing approaches for outline extraction from aerial or satellite imagery are typically decomposed into subtasks, e.g., building masking and vectorization, or treat this task as a sequence-to-sequence prediction of ordered vertices. The former lacks efficiency, and the latter often generates redundant vertices, both resulting in suboptimal performance. To handle these issues, we propose a novel Region-of-Interest (RoI) query-based approach called RoIPoly. Specifically, we formulate each vertex as a query and constrain the query attention on the most relevant regions of a potential building, yielding reduced computational overhead and more efficient vertex level interaction. Moreover, we introduce a novel learnable logit embedding to facilitate vertex classification on the attention map; thus, no post-processing is needed for redundant vertex removal. We evaluated our method on the vectorized building outline extraction dataset CrowdAI and the 2D floorplan reconstruction dataset Structured3D. On the CrowdAI dataset, RoIPoly with a ResNet50 backbone outperforms existing methods with the same or better backbones on most MS-COCO metrics, especially on small buildings, and achieves competitive results in polygon quality and vertex redundancy without any post-processing. On the Structured3D dataset, our method achieves the second-best performance on most metrics among existing methods dedicated to 2D floorplan reconstruction, demonstrating our cross-domain generalization capability. The code will be released upon acceptance of this paper.