Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration
This work addresses the challenge of making low-cost robots capable of multi-task manipulation, though it appears to be an incremental improvement combining existing techniques like VAEs, GANs, and behavior cloning.
The authors tackled the problem of enabling inexpensive robots to perform complex manipulation tasks by developing an end-to-end learning from demonstration approach that uses raw images as input and generates robot arm trajectories. Their results showed that weight sharing and reconstruction-based regularization improved generalization and robustness, with training on multiple tasks simultaneously increasing success rates across all tasks.
We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation. The controller is a recurrent neural network using raw images as input and generating robot arm trajectories, with the parameters shared across the tasks. The controller also combines VAE-GAN-based reconstruction with autoregressive multimodal action prediction. Our results demonstrate that it is possible to learn complex manipulation tasks, such as picking up a towel, wiping an object, and depositing the towel to its previous position, entirely from raw images with direct behavior cloning. We show that weight sharing and reconstruction-based regularization substantially improve generalization and robustness, and training on multiple tasks simultaneously increases the success rate on all tasks.