Rethinking Score Distilling Sampling for 3D Editing and Generation
This addresses the problem of separate methods for 3D generation and editing for researchers and practitioners in 3D AI, offering a unified solution, though it appears incremental as it refines existing gradient terms.
The paper tackled the limitation of Score Distillation Sampling (SDS) in 3D tasks, which could only generate but not edit assets, by proposing Unified Distillation Sampling (UDS) that unifies generation and editing, resulting in outperforming baselines with richer details in generation and excelling in editing.
Score Distillation Sampling (SDS) has emerged as a prominent method for text-to-3D generation by leveraging the strengths of 2D diffusion models. However, SDS is limited to generation tasks and lacks the capability to edit existing 3D assets. Conversely, variants of SDS that introduce editing capabilities often can not generate new 3D assets effectively. In this work, we observe that the processes of generation and editing within SDS and its variants have unified underlying gradient terms. Building on this insight, we propose Unified Distillation Sampling (UDS), a method that seamlessly integrates both the generation and editing of 3D assets. Essentially, UDS refines the gradient terms used in vanilla SDS methods, unifying them to support both tasks. Extensive experiments demonstrate that UDS not only outperforms baseline methods in generating 3D assets with richer details but also excels in editing tasks, thereby bridging the gap between 3D generation and editing. The code is available on: https://github.com/xingy038/UDS.