GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis
This work addresses the challenge of generating and reconstructing structurally complex 3D shapes for applications in computer graphics and vision, representing an incremental improvement over existing neural field formulations.
The authors tackled the problem of 3D shape synthesis and reconstruction by introducing GEM3D, a generative model that uses neural skeleton-based representations and denoising diffusion to produce more topologically and geometrically accurate surfaces, achieving significantly more faithful reconstruction and diverse generation compared to state-of-the-art methods on datasets like Thingi10K and ShapeNet.
We introduce GEM3D -- a new deep, topology-aware generative model of 3D shapes. The key ingredient of our method is a neural skeleton-based representation encoding information on both shape topology and geometry. Through a denoising diffusion probabilistic model, our method first generates skeleton-based representations following the Medial Axis Transform (MAT), then generates surfaces through a skeleton-driven neural implicit formulation. The neural implicit takes into account the topological and geometric information stored in the generated skeleton representations to yield surfaces that are more topologically and geometrically accurate compared to previous neural field formulations. We discuss applications of our method in shape synthesis and point cloud reconstruction tasks, and evaluate our method both qualitatively and quantitatively. We demonstrate significantly more faithful surface reconstruction and diverse shape generation results compared to the state-of-the-art, also involving challenging scenarios of reconstructing and synthesizing structurally complex, high-genus shape surfaces from Thingi10K and ShapeNet.