ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation
This addresses a key bottleneck in robotic manipulation for transparent objects, which are common in everyday environments but difficult for standard sensors, offering a practical solution with potential real-world applications.
The paper tackles the problem of estimating accurate 3D geometry for transparent objects from single RGB-D images, which is challenging due to their visual properties, and presents ClearGrasp, a deep learning approach that improves depth estimates and generalizes to real-world images and novel objects, with experiments showing it substantially outperforms baselines and enhances grasping performance.
Transparent objects are a common part of everyday life, yet they possess unique visual properties that make them incredibly difficult for standard 3D sensors to produce accurate depth estimates for. In many cases, they often appear as noisy or distorted approximations of the surfaces that lie behind them. To address these challenges, we present ClearGrasp -- a deep learning approach for estimating accurate 3D geometry of transparent objects from a single RGB-D image for robotic manipulation. Given a single RGB-D image of transparent objects, ClearGrasp uses deep convolutional networks to infer surface normals, masks of transparent surfaces, and occlusion boundaries. It then uses these outputs to refine the initial depth estimates for all transparent surfaces in the scene. To train and test ClearGrasp, we construct a large-scale synthetic dataset of over 50,000 RGB-D images, as well as a real-world test benchmark with 286 RGB-D images of transparent objects and their ground truth geometries. The experiments demonstrate that ClearGrasp is substantially better than monocular depth estimation baselines and is capable of generalizing to real-world images and novel objects. We also demonstrate that ClearGrasp can be applied out-of-the-box to improve grasping algorithms' performance on transparent objects. Code, data, and benchmarks will be released. Supplementary materials available on the project website: https://sites.google.com/view/cleargrasp