Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories
This addresses a bottleneck in 3D editing for applications requiring extensive modifications, though it appears incremental as it builds on existing diffusion models and NeRF frameworks.
The paper tackles the problem of limited geometric or appearance changes in existing 3D editing methods by proposing Perturb-and-Revise, which enables flexible and effective editing of color, appearance, and geometry in 3D NeRFs.
Recent advancements in text-based diffusion models have accelerated progress in 3D reconstruction and text-based 3D editing. Although existing 3D editing methods excel at modifying color, texture, and style, they struggle with extensive geometric or appearance changes, thus limiting their applications. To this end, we propose Perturb-and-Revise, which makes possible a variety of NeRF editing. First, we perturb the NeRF parameters with random initializations to create a versatile initialization. The level of perturbation is determined automatically through analysis of the local loss landscape. Then, we revise the edited NeRF via generative trajectories. Combined with the generative process, we impose identity-preserving gradients to refine the edited NeRF. Extensive experiments demonstrate that Perturb-and-Revise facilitates flexible, effective, and consistent editing of color, appearance, and geometry in 3D. For 360° results, please visit our project page: https://susunghong.github.io/Perturb-and-Revise.