Geometry Aware Field-to-field Transformations for 3D Semantic Segmentation
This addresses the problem of 3D semantic segmentation for computer vision researchers by providing a method that works with any NeRF scene parameterization, though it appears incremental as it builds on existing NeRF and autoencoding techniques.
The paper tackles 3D semantic segmentation using only 2D supervision by leveraging Neural Radiance Fields (NeRFs) to extract features from surface point clouds, achieving a compact and sample-efficient representation that enables few-shot segmentation.
We present a novel approach to perform 3D semantic segmentation solely from 2D supervision by leveraging Neural Radiance Fields (NeRFs). By extracting features along a surface point cloud, we achieve a compact representation of the scene which is sample-efficient and conducive to 3D reasoning. Learning this feature space in an unsupervised manner via masked autoencoding enables few-shot segmentation. Our method is agnostic to the scene parameterization, working on scenes fit with any type of NeRF.