BuildAnyPoint: 3D Building Structured Abstraction from Diverse Point Clouds
This work addresses the challenge of generating structured 3D building models from noisy or sparse point cloud data, which is important for applications in architecture, urban planning, and computer vision, but it appears incremental as it builds on existing generative priors and autoregressive methods.
The paper tackles the problem of structured 3D building reconstruction from diverse point clouds, such as airborne LiDAR and Structure-from-Motion, by introducing BuildAnyPoint, a generative framework that recovers artist-created building abstractions and delivers substantial qualitative and quantitative improvements over prior methods, with evidence of strong performance on building point cloud completion benchmarks.
We introduce BuildAnyPoint, a novel generative framework for structured 3D building reconstruction from point clouds with diverse distributions, such as those captured by airborne LiDAR and Structure-from-Motion. To recover artist-created building abstraction in this highly underconstrained setting, we capitalize on the role of explicit 3D generative priors in autoregressive mesh generation. Specifically, we design a Loosely Cascaded Diffusion Transformer (Loca-DiT) that initially recovers the underlying distribution from noisy or sparse points, followed by autoregressively encapsulating them into compact meshes. We first formulate distribution recovery as a conditional generation task by training latent diffusion models conditioned on input point clouds, and then tailor a decoder-only transformer for conditional autoregressive mesh generation based on the recovered point clouds. Our method delivers substantial qualitative and quantitative improvements over prior building abstraction methods. Furthermore, the effectiveness of our approach is evidenced by the strong performance of its recovered point clouds on building point cloud completion benchmarks, which exhibit improved surface accuracy and distribution uniformity.