FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling
This addresses the problem of 3D object modeling from images for computer vision applications, offering an incremental improvement by combining foreground-background separation with NeRF.
The paper tackles learning 3D object category models from images by using a 2-component NeRF to separate foreground objects from varying backgrounds, achieving accurate modeling and amodal segmentation with photometric supervision and casually captured images.
We investigate the use of Neural Radiance Fields (NeRF) to learn high quality 3D object category models from collections of input images. In contrast to previous work, we are able to do this whilst simultaneously separating foreground objects from their varying backgrounds. We achieve this via a 2-component NeRF model, FiG-NeRF, that prefers explanation of the scene as a geometrically constant background and a deformable foreground that represents the object category. We show that this method can learn accurate 3D object category models using only photometric supervision and casually captured images of the objects. Additionally, our 2-part decomposition allows the model to perform accurate and crisp amodal segmentation. We quantitatively evaluate our method with view synthesis and image fidelity metrics, using synthetic, lab-captured, and in-the-wild data. Our results demonstrate convincing 3D object category modelling that exceed the performance of existing methods.