ROI-NeRFs: Hi-Fi Visualization of Objects of Interest within a Scene by NeRFs Composition
This work addresses the problem of efficient and accurate 3D reconstruction for cultural heritage applications, representing an incremental improvement by focusing on user-defined objects of interest.
The study tackled the challenge of visualizing objects within large-scale scenes at high detail using Neural Radiance Fields (NeRFs), resulting in improved level of detail for object regions and minimized artifacts without significantly increasing inference time, as demonstrated on real-world datasets including a complex cultural heritage room.
Efficient and accurate 3D reconstruction is essential for applications in cultural heritage. This study addresses the challenge of visualizing objects within large-scale scenes at a high level of detail (LOD) using Neural Radiance Fields (NeRFs). The aim is to improve the visual fidelity of chosen objects while maintaining the efficiency of the computations by focusing on details only for relevant content. The proposed ROI-NeRFs framework divides the scene into a Scene NeRF, which represents the overall scene at moderate detail, and multiple ROI NeRFs that focus on user-defined objects of interest. An object-focused camera selection module automatically groups relevant cameras for each NeRF training during the decomposition phase. In the composition phase, a Ray-level Compositional Rendering technique combines information from the Scene NeRF and ROI NeRFs, allowing simultaneous multi-object rendering composition. Quantitative and qualitative experiments conducted on two real-world datasets, including one on a complex eighteen's century cultural heritage room, demonstrate superior performance compared to baseline methods, improving LOD for object regions, minimizing artifacts, and without significantly increasing inference time.