ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsampling
This addresses the problem of creating detailed 3D architectural models for designers and architects, representing a novel method for a known bottleneck in 3D generation.
The paper tackles the challenge of generating complex 3D architectural designs with fine details by introducing ArchComplete, a two-stage voxel-based pipeline that autoregressively generates coarse shapes and refines them to high resolutions up to 512^3, achieving state-of-the-art performance in quality, diversity, and efficiency.
Recent advances in 3D generative models have shown promising results but often fall short in capturing the complexity of architectural geometries and topologies and fine geometric details at high resolutions. To tackle this, we present ArchComplete, a two-stage voxel-based 3D generative pipeline consisting of a vector-quantised model, whose composition is modelled with an autoregressive transformer for generating coarse shapes, followed by a hierarchical upsampling strategy for further enrichment with fine structures and details. Key to our pipeline is (i) learning a contextually rich codebook of local patch embeddings, optimised alongside a 2.5D perceptual loss that captures global spatial correspondence of projections onto three axis-aligned orthogonal planes, and (ii) redefining upsampling as a set of conditional diffusion models learning from a hierarchy of randomly cropped coarse-to-fine local volumetric patches. Trained on our introduced dataset of 3D house models with fully modelled exterior and interior, ArchComplete autoregressively generates models at the resolution of $64^{3}$ and progressively refines them up to $512^{3}$, with voxel sizes as small as $ \approx 9\text{cm}$. ArchComplete solves a variety of tasks, including genetic interpolation and variation, unconditional synthesis, shape and plan-drawing completion, as well as geometric detailisation, while achieving state-of-the-art performance in quality, diversity, and computational efficiency.