EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points
This addresses the need for user-friendly editing tools in dynamic 3D scene reconstruction, particularly for applications in graphics and VR, though it is an incremental improvement over existing NeRF-based methods.
The paper tackles the problem of editing dynamic scenes modeled by Neural Radiance Fields (NeRF), which is challenging due to topological changes, and proposes EditableNeRF, a method that enables intuitive editing via key points, achieving high-quality results and outperforming state-of-the-art approaches.
Neural radiance fields (NeRF) achieve highly photo-realistic novel-view synthesis, but it's a challenging problem to edit the scenes modeled by NeRF-based methods, especially for dynamic scenes. We propose editable neural radiance fields that enable end-users to easily edit dynamic scenes and even support topological changes. Input with an image sequence from a single camera, our network is trained fully automatically and models topologically varying dynamics using our picked-out surface key points. Then end-users can edit the scene by easily dragging the key points to desired new positions. To achieve this, we propose a scene analysis method to detect and initialize key points by considering the dynamics in the scene, and a weighted key points strategy to model topologically varying dynamics by joint key points and weights optimization. Our method supports intuitive multi-dimensional (up to 3D) editing and can generate novel scenes that are unseen in the input sequence. Experiments demonstrate that our method achieves high-quality editing on various dynamic scenes and outperforms the state-of-the-art. Our code and captured data are available at https://chengwei-zheng.github.io/EditableNeRF/.