Multi-modal Transfer Learning for Grasping Transparent and Specular Objects
This addresses a specific challenge in robotics for handling objects like glass or mirrors, but it is incremental as it builds on existing grasping methods.
The paper tackled the problem of grasping transparent and specular objects, which are undetectable by standard depth sensors, by introducing a multi-modal transfer learning method that bootstraps from an existing uni-modal model, resulting in reliable grasping without requiring ground-truth labels or real grasp attempts.
State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects. To improve grasping performance on such objects, we introduce a method for learning a multi-modal perception model by bootstrapping from an existing uni-modal model. This transfer learning approach requires only a pre-existing uni-modal grasping model and paired multi-modal image data for training, foregoing the need for ground-truth grasp success labels nor real grasp attempts. Our experiments demonstrate that our approach is able to reliably grasp transparent and reflective objects. Video and supplementary material are available at https://sites.google.com/view/transparent-specular-grasping.