Improved Reinforcement Learning Pushing Policies via Heuristic Rules
This work addresses a specific problem in robotics for object manipulation, but it is incremental as it builds on existing reinforcement learning methods with a heuristic enhancement.
The paper tackled the problem of robotic singulation of target objects from clutter using non-prehensile pushing actions, and the result was that incorporating a heuristic rule into reinforcement learning reward functions improved performance, as shown in simulation experiments.
Non-prehensile pushing actions have the potential to singulate a target object from its surrounding clutter in order to facilitate the robotic grasping of the target. To address this problem we utilize a heuristic rule that moves the target object towards the workspace's empty space and demonstrate that this simple heuristic rule achieves singulation. We incorporate this effective heuristic rule to the reward in order to train more efficiently reinforcement learning (RL) agents for singulation. Simulation experiments demonstrate that this insight increases performance. Finally, our results show that the RL-based policy implicitly learns something similar to one of the used heuristics in terms of decision making. Qualitative results, code, pre-trained models and simulation environments are available at https://github.com/robot-clutter/improved_rl.