A Mesh Is Worth 512 Numbers: Spectral-domain Diffusion Modeling for High-dimension Shape Generation
This work addresses the problem of efficient 3D generative modeling for applications with limited data or GPU resources, though it is incremental as it builds on existing spectral and diffusion methods.
The paper tackles the computational expense of learning latent codes for 3D shape generation by introducing SpoDify, a framework that uses singular value decomposition (SVD) for encoding meshes into 512-dimensional latent codes without training, achieving high-quality shape generation comparable to state-of-the-art methods.
Recent advancements in learning latent codes derived from high-dimensional shapes have demonstrated impressive outcomes in 3D generative modeling. Traditionally, these approaches employ a trained autoencoder to acquire a continuous implicit representation of source shapes, which can be computationally expensive. This paper introduces a novel framework, spectral-domain diffusion for high-quality shape generation SpoDify, that utilizes singular value decomposition (SVD) for shape encoding. The resulting eigenvectors can be stored for subsequent decoding, while generative modeling is performed on the eigenfeatures. This approach efficiently encodes complex meshes into continuous implicit representations, such as encoding a 15k-vertex mesh to a 512-dimensional latent code without learning. Our method exhibits significant advantages in scenarios with limited samples or GPU resources. In mesh generation tasks, our approach produces high-quality shapes that are comparable to state-of-the-art methods.