DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF
This addresses the need for flexible 3D scene understanding in robotics and autonomous systems by combining open-vocabulary and unsupervised approaches, though it appears incremental as it builds on existing neural field representations.
The paper tackles the problem of 3D semantic segmentation by proposing DiSCO-3D, which adapts to both scene content and user queries for open-vocabulary sub-concept discovery, achieving state-of-the-art results in edge cases of open-vocabulary and unsupervised segmentation.
3D semantic segmentation provides high-level scene understanding for applications in robotics, autonomous systems, \textit{etc}. Traditional methods adapt exclusively to either task-specific goals (open-vocabulary segmentation) or scene content (unsupervised semantic segmentation). We propose DiSCO-3D, the first method addressing the broader problem of 3D Open-Vocabulary Sub-concepts Discovery, which aims to provide a 3D semantic segmentation that adapts to both the scene and user queries. We build DiSCO-3D on Neural Fields representations, combining unsupervised segmentation with weak open-vocabulary guidance. Our evaluations demonstrate that DiSCO-3D achieves effective performance in Open-Vocabulary Sub-concepts Discovery and exhibits state-of-the-art results in the edge cases of both open-vocabulary and unsupervised segmentation.