Synthesizing 3D Abstractions by Inverting Procedural Buildings with Transformers
This provides a method for creating efficient, regular building abstractions from point clouds, primarily benefiting gaming and animation applications.
The paper tackles the problem of generating abstract 3D building models by learning to invert procedural models using transformers, achieving good reconstruction accuracy and structurally consistent inpainting.
We generate abstractions of buildings, reflecting the essential aspects of their geometry and structure, by learning to invert procedural models. We first build a dataset of abstract procedural building models paired with simulated point clouds and then learn the inverse mapping through a transformer. Given a point cloud, the trained transformer then infers the corresponding abstracted building in terms of a programmatic language description. This approach leverages expressive procedural models developed for gaming and animation, and thereby retains desirable properties such as efficient rendering of the inferred abstractions and strong priors for regularity and symmetry. Our approach achieves good reconstruction accuracy in terms of geometry and structure, as well as structurally consistent inpainting.