SIn-NeRF2NeRF: Editing 3D Scenes with Instructions through Segmentation and Inpainting
This work addresses a specific limitation in 3D scene editing for applications like computer graphics and virtual reality, but it is incremental as it builds on existing methods like Instruct-NeRF2NeRF.
The paper tackles the challenge of performing geometrical modifications like resizing or moving objects in 3D scenes represented by Neural Radiance Fields (NeRF), by selectively editing objects after separating them from the background using segmentation and inpainting, and demonstrates examples of these edits.
TL;DR Perform 3D object editing selectively by disentangling it from the background scene. Instruct-NeRF2NeRF (in2n) is a promising method that enables editing of 3D scenes composed of Neural Radiance Field (NeRF) using text prompts. However, it is challenging to perform geometrical modifications such as shrinking, scaling, or moving on both the background and object simultaneously. In this project, we enable geometrical changes of objects within the 3D scene by selectively editing the object after separating it from the scene. We perform object segmentation and background inpainting respectively, and demonstrate various examples of freely resizing or moving disentangled objects within the three-dimensional space.