Semantic Is Enough: Only Semantic Information For NeRF Reconstruction
This work is incremental, as it simplifies an existing method for researchers in 3D computer vision by focusing on semantic-only inputs.
The paper tackled the problem of 3D scene reconstruction by modifying Semantic-NeRF to use only semantic information without RGB data, and found that this approach still enables effective scene understanding tasks like object detection and segmentation.
Recent research that combines implicit 3D representation with semantic information, like Semantic-NeRF, has proven that NeRF model could perform excellently in rendering 3D structures with semantic labels. This research aims to extend the Semantic Neural Radiance Fields (Semantic-NeRF) model by focusing solely on semantic output and removing the RGB output component. We reformulate the model and its training procedure to leverage only the cross-entropy loss between the model semantic output and the ground truth semantic images, removing the colour data traditionally used in the original Semantic-NeRF approach. We then conduct a series of identical experiments using the original and the modified Semantic-NeRF model. Our primary objective is to obverse the impact of this modification on the model performance by Semantic-NeRF, focusing on tasks such as scene understanding, object detection, and segmentation. The results offer valuable insights into the new way of rendering the scenes and provide an avenue for further research and development in semantic-focused 3D scene understanding.