ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points
This addresses the challenge of 3D reconstruction for architectural modeling, though it appears incremental as it builds on existing procedural modeling and learning techniques.
ArcPro tackles the problem of recovering structured 3D abstractions from sparse and low-quality point clouds by using architectural programs, achieving superior performance over traditional and learning-based methods in experiments.
We introduce ArcPro, a novel learning framework built on architectural programs to recover structured 3D abstractions from highly sparse and low-quality point clouds. Specifically, we design a domain-specific language (DSL) to hierarchically represent building structures as a program, which can be efficiently converted into a mesh. We bridge feedforward and inverse procedural modeling by using a feedforward process for training data synthesis, allowing the network to make reverse predictions. We train an encoder-decoder on the points-program pairs to establish a mapping from unstructured point clouds to architectural programs, where a 3D convolutional encoder extracts point cloud features and a transformer decoder autoregressively predicts the programs in a tokenized form. Inference by our method is highly efficient and produces plausible and faithful 3D abstractions. Comprehensive experiments demonstrate that ArcPro outperforms both traditional architectural proxy reconstruction and learning-based abstraction methods. We further explore its potential to work with multi-view image and natural language inputs.