On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation
This addresses the challenge of robust policy learning for robotic manipulation in real-world scenarios, but it is incremental as it builds on existing methods for specific applications.
The authors tackled the problem of learning in-hand manipulation skills with a prosthetic hand while minimizing irreversible events like object slipping, by combining reinforcement learning with tactile reactive control, resulting in significantly reduced slipping and successful task completion.
In this letter, we present an approach for learning in-hand manipulation skills with a low-cost, underactuated prosthetic hand in the presence of irreversible events. Our approach combines reinforcement learning based on visual perception with low-level reactive control based on tactile perception, which aims to avoid slipping. The objective of the reinforcement learning level consists not only in fulfilling the in-hand manipulation goal, but also in minimizing the intervention of the tactile reactive control. This way, the occurrence of object slipping during the learning procedure, which we consider an irreversible event, is significantly reduced. When an irreversible event occurs, the learning process is considered failed. We show the performance in two tasks, which consist in reorienting a cup and a bottle only using the fingers. The experimental results show that the proposed architecture allows reaching the goal in the Cartesian space and reduces significantly the occurrence of object slipping during the learning procedure. Moreover, without the proposed synergy between reactive control and reinforcement learning it was not possible to avoid irreversible events and, therefore, to learn the task.