SDM-NET: Deep Generative Network for Structured Deformable Mesh
This work addresses the challenge of generating realistic and structured 3D shapes for applications like computer graphics and modeling, though it appears incremental as it builds on existing VAE frameworks.
The authors tackled the problem of generating structured 3D meshes by introducing SDM-NET, a deep generative network that decomposes shapes into deformable parts and jointly learns part geometries and global structure, resulting in superior visual quality and flexible topology compared to state-of-the-art models.
We introduce SDM-NET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed, deformable mesh parts, which respect the global part structure of a shape collection, e.g., chairs, airplanes, etc. Our key observation is that while the overall structure of a 3D shape can be complex, the shape can usually be decomposed into a set of parts, each homeomorphic to a box, and the finer-scale geometry of the part can be recovered by deforming the box. The architecture of SDM-NET is that of a two-level variational autoencoder (VAE). At the part level, a PartVAE learns a deformable model of part geometries. At the structural level, we train a Structured Parts VAE (SP-VAE), which jointly learns the part structure of a shape collection and the part geometries, ensuring a coherence between global shape structure and surface details. Through extensive experiments and comparisons with the state-of-the-art deep generative models of shapes, we demonstrate the superiority of SDM-NET in generating meshes with visual quality, flexible topology, and meaningful structures, which benefit shape interpolation and other subsequently modeling tasks.