Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space
This work addresses the problem of inefficient and unsafe robotic manipulation for real-world applications, representing an incremental improvement by integrating existing methods.
The paper tackles the challenge of robotic manipulation by combining slow reinforcement learning with fast adaptive control through a bio-inspired action space called AFORCE, resulting in improved sample efficiency, reduced energy consumption, and enhanced safety in contact-rich tasks.
Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in real-world robotics, these methods still struggle, as they require large amounts of expensive interactions and have slow feedback loops. On the other hand, fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. In this work, we propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds. The framework consists of two components, a slow reinforcement learning policy optimizing the task strategy given multimodal observations, and a fast, real-time adaptive control policy continuously optimizing the motion, stability, and effort of the manipulator. We combine these components through a bio-inspired action space that we call AFORCE. We demonstrate the new action space on a contact-rich manipulation task on real hardware and evaluate its performance on three simulated manipulation tasks. Our experiments show that AFORCE drastically improves sample efficiency while reducing energy consumption and improving safety.