GrabS: Generative Embodied Agent for 3D Object Segmentation without Scene Supervision
This addresses the challenge of unsupervised 3D object segmentation for robotics and computer vision applications, representing a novel method rather than an incremental improvement.
The paper tackles the problem of 3D object segmentation in complex point clouds without human-labeled scene supervision, achieving remarkable segmentation performance that surpasses all existing unsupervised methods on real-world and synthetic datasets.
We study the hard problem of 3D object segmentation in complex point clouds without requiring human labels of 3D scenes for supervision. By relying on the similarity of pretrained 2D features or external signals such as motion to group 3D points as objects, existing unsupervised methods are usually limited to identifying simple objects like cars or their segmented objects are often inferior due to the lack of objectness in pretrained features. In this paper, we propose a new two-stage pipeline called GrabS. The core concept of our method is to learn generative and discriminative object-centric priors as a foundation from object datasets in the first stage, and then design an embodied agent to learn to discover multiple objects by querying against the pretrained generative priors in the second stage. We extensively evaluate our method on two real-world datasets and a newly created synthetic dataset, demonstrating remarkable segmentation performance, clearly surpassing all existing unsupervised methods.