ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context
This work addresses the need for more accessible and efficient editing tools for NeRFs, which are crucial for applications in 3D content creation and visualization, though it appears incremental as it builds on existing NeRF frameworks.
The paper tackles the problem of limited interactive editing options for Neural Radiance Fields (NeRFs) by introducing a fast and efficient neural network architecture that enables user-friendly, view-consistent image-based edits, achieving a 10-30x speedup over concurrent text-guided NeRF editing methods.
Neural Radiance Fields (NeRFs) have recently emerged as a popular option for photo-realistic object capture due to their ability to faithfully capture high-fidelity volumetric content even from handheld video input. Although much research has been devoted to efficient optimization leading to real-time training and rendering, options for interactive editing NeRFs remain limited. We present a very simple but effective neural network architecture that is fast and efficient while maintaining a low memory footprint. This architecture can be incrementally guided through user-friendly image-based edits. Our representation allows straightforward object selection via semantic feature distillation at the training stage. More importantly, we propose a local 3D-aware image context to facilitate view-consistent image editing that can then be distilled into fine-tuned NeRFs, via geometric and appearance adjustments. We evaluate our setup on a variety of examples to demonstrate appearance and geometric edits and report 10-30x speedup over concurrent work focusing on text-guided NeRF editing. Video results can be seen on our project webpage at https://proteusnerf.github.io.