DORec: Decomposed Object Reconstruction and Segmentation Utilizing 2D Self-Supervised Features
This work addresses the challenge of object decomposition for robotics applications such as manipulation and autonomous driving, offering an incremental improvement by leveraging self-supervised features to reduce reliance on manual labels.
The paper tackled the problem of decomposing and reconstructing 3D objects from complex backgrounds by proposing DORec, a network that uses 2D self-supervised features to create masks for supervision, resulting in robust decomposition and enabling downstream tasks like pose estimation.
Recovering 3D geometry and textures of individual objects is crucial for many robotics applications, such as manipulation, pose estimation, and autonomous driving. However, decomposing a target object from a complex background is challenging. Most existing approaches rely on costly manual labels to acquire object instance perception. Recent advancements in 2D self-supervised learning offer new prospects for identifying objects of interest, yet leveraging such noisy 2D features for clean decomposition remains difficult. In this paper, we propose a Decomposed Object Reconstruction (DORec) network based on neural implicit representations. Our key idea is to use 2D self-supervised features to create two levels of masks for supervision: a binary mask for foreground regions and a K-cluster mask for semantically similar regions. These complementary masks result in robust decomposition. Experimental results on different datasets show DORec's superiority in segmenting and reconstructing diverse foreground objects from varied backgrounds enabling downstream tasks such as pose estimation.