Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions
This work addresses the challenge of 3D shape generation for applications like computer graphics and robotics, representing an incremental advancement by extending diffusion models to implicit 3D representations.
The paper tackles the problem of generating complex 3D shapes using diffusion models, which have been limited to 2D tasks, by proposing Diffusion-SDF for shape completion and reconstruction, achieving realistic unconditional and conditional generation as shown in experiments.
Probabilistic diffusion models have achieved state-of-the-art results for image synthesis, inpainting, and text-to-image tasks. However, they are still in the early stages of generating complex 3D shapes. This work proposes Diffusion-SDF, a generative model for shape completion, single-view reconstruction, and reconstruction of real-scanned point clouds. We use neural signed distance functions (SDFs) as our 3D representation to parameterize the geometry of various signals (e.g., point clouds, 2D images) through neural networks. Neural SDFs are implicit functions and diffusing them amounts to learning the reversal of their neural network weights, which we solve using a custom modulation module. Extensive experiments show that our method is capable of both realistic unconditional generation and conditional generation from partial inputs. This work expands the domain of diffusion models from learning 2D, explicit representations, to 3D, implicit representations.