ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping
This addresses the issue of suboptimal motion and collisions in robotic grasping for embodied systems, representing a strong specific gain rather than a broad paradigm shift.
The paper tackles the problem of robotic grasping by introducing ZeroGrasp, a framework that simultaneously performs 3D reconstruction and grasp pose prediction in near real-time, achieving state-of-the-art performance on the GraspNet-1B benchmark and generalizing to novel real-world objects.
Robotic grasping is a cornerstone capability of embodied systems. Many methods directly output grasps from partial information without modeling the geometry of the scene, leading to suboptimal motion and even collisions. To address these issues, we introduce ZeroGrasp, a novel framework that simultaneously performs 3D reconstruction and grasp pose prediction in near real-time. A key insight of our method is that occlusion reasoning and modeling the spatial relationships between objects is beneficial for both accurate reconstruction and grasping. We couple our method with a novel large-scale synthetic dataset, which comprises 1M photo-realistic images, high-resolution 3D reconstructions and 11.3B physically-valid grasp pose annotations for 12K objects from the Objaverse-LVIS dataset. We evaluate ZeroGrasp on the GraspNet-1B benchmark as well as through real-world robot experiments. ZeroGrasp achieves state-of-the-art performance and generalizes to novel real-world objects by leveraging synthetic data.