Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation
This addresses the challenging problem of deformable object manipulation for robotics, but it is incremental as it builds on existing learning-based methods.
The paper tackles the problem of vision-based rope manipulation in robotics by combining self-supervised learning and imitation, enabling a robot to reproduce human demonstrations using only monocular images, with results showing successful manipulation into various target shapes.
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics. We present a learning-based system where a robot takes as input a sequence of images of a human manipulating a rope from an initial to goal configuration, and outputs a sequence of actions that can reproduce the human demonstration, using only monocular images as input. To perform this task, the robot learns a pixel-level inverse dynamics model of rope manipulation directly from images in a self-supervised manner, using about 60K interactions with the rope collected autonomously by the robot. The human demonstration provides a high-level plan of what to do and the low-level inverse model is used to execute the plan. We show that by combining the high and low-level plans, the robot can successfully manipulate a rope into a variety of target shapes using only a sequence of human-provided images for direction.