Learning to Build Shapes by Extrusion
This addresses the problem of generating and editing 3D shapes for artists and designers, offering an incremental improvement over transformer-based models by mimicking artistic workflows.
The paper tackles 3D mesh generation by introducing Text Encoded Extrusion (TEE), a text-based representation using sequences of face extrusions, and trains a large language model to generate and edit meshes, producing manifold meshes with arbitrary face counts.
We introduce Text Encoded Extrusion (TEE), a text-based representation that expresses mesh construction as sequences of face extrusions rather than polygon lists, and a method for generating 3D meshes from TEE using a large language model (LLM). By learning extrusion sequences that assemble a mesh, similar to the way artists create meshes, our approach naturally supports arbitrary output face counts and produces manifold meshes by design, in contrast to recent transformer-based models. The learnt extrusion sequences can also be applied to existing meshes - enabling editing in addition to generation. To train our model, we decompose a library of quadrilateral meshes with non-self-intersecting face loops into constituent loops, which can be viewed as their building blocks, and finetune an LLM on the steps for reassembling the meshes by performing a sequence of extrusions. We demonstrate that our representation enables reconstruction, novel shape synthesis, and the addition of new features to existing meshes.