TreeMeshGPT: Artistic Mesh Generation with Autoregressive Tree Sequencing
This work addresses the challenge of generating detailed and consistent artistic meshes for applications in computer graphics and 3D modeling, representing an incremental improvement over existing methods.
The authors tackled the problem of generating high-quality artistic meshes from point clouds by introducing TreeMeshGPT, which uses a novel Autoregressive Tree Sequencing method to reduce training difficulty and improve mesh quality, achieving a 22% compression rate in tokenization and generating meshes with refined details and consistent normal orientation.
We introduce TreeMeshGPT, an autoregressive Transformer designed to generate high-quality artistic meshes aligned with input point clouds. Instead of the conventional next-token prediction in autoregressive Transformer, we propose a novel Autoregressive Tree Sequencing where the next input token is retrieved from a dynamically growing tree structure that is built upon the triangle adjacency of faces within the mesh. Our sequencing enables the mesh to extend locally from the last generated triangular face at each step, and therefore reduces training difficulty and improves mesh quality. Our approach represents each triangular face with two tokens, achieving a compression rate of approximately 22% compared to the naive face tokenization. This efficient tokenization enables our model to generate highly detailed artistic meshes with strong point cloud conditioning, surpassing previous methods in both capacity and fidelity. Furthermore, our method generates mesh with strong normal orientation constraints, minimizing flipped normals commonly encountered in previous methods. Our experiments show that TreeMeshGPT enhances the mesh generation quality with refined details and normal orientation consistency.