Neural Implicit Field Editing Considering Object-environment Interaction
This work addresses a specific limitation in 3D scene editing for applications like virtual reality or graphics, but it is incremental as it builds on existing neural implicit field methods.
The paper tackles the problem of 3D scene editing with neural implicit fields by addressing the blending of object-environment interactions, such as shadows, which existing methods fail to display. The proposed OSI-aware system uses a two-stream neural rendering approach with intrinsic decomposition and achieves reasonable appearance changes in editing tasks and competitive performance in novel-view synthesis.
The 3D scene editing method based on neural implicit field has gained wide attention. It has achieved excellent results in 3D editing tasks. However, existing methods often blend the interaction between objects and scene environment. The change of scene appearance like shadows is failed to be displayed in the rendering view. In this paper, we propose an Object and Scene environment Interaction aware (OSI-aware) system, which is a novel two-stream neural rendering system considering object and scene environment interaction. To obtain illuminating conditions from the mixture soup, the system successfully separates the interaction between objects and scene environment by intrinsic decomposition method. To study the corresponding changes to the scene appearance from object-level editing tasks, we introduce a depth map guided scene inpainting method and shadow rendering method by point matching strategy. Extensive experiments demonstrate that our novel pipeline produce reasonable appearance changes in scene editing tasks. It also achieve competitive performance for the rendering quality in novel-view synthesis tasks.