Learning Force Control for Contact-rich Manipulation Tasks with Rigid Position-controlled Robots
This work addresses the problem of safely and effectively using RL on real robotic hardware for researchers and practitioners in robotics, though it is incremental as it builds on existing control methods.
The paper tackled the challenge of applying reinforcement learning to real rigid position-controlled robots for contact-rich manipulation tasks by proposing a learning-based force control framework that combines RL with traditional force control methods, achieving successful validation in simulation and on a UR3 robotic arm.
Reinforcement Learning (RL) methods have been proven successful in solving manipulation tasks autonomously. However, RL is still not widely adopted on real robotic systems because working with real hardware entails additional challenges, especially when using rigid position-controlled manipulators. These challenges include the need for a robust controller to avoid undesired behavior, that risk damaging the robot and its environment, and constant supervision from a human operator. The main contributions of this work are, first, we proposed a learning-based force control framework combining RL techniques with traditional force control. Within said control scheme, we implemented two different conventional approaches to achieve force control with position-controlled robots; one is a modified parallel position/force control, and the other is an admittance control. Secondly, we empirically study both control schemes when used as the action space of the RL agent. Thirdly, we developed a fail-safe mechanism for safely training an RL agent on manipulation tasks using a real rigid robot manipulator. The proposed methods are validated on simulation and a real robot, an UR3 e-series robotic arm.