3D Simulation for Robot Arm Control with Deep Q-Learning
This addresses the data efficiency challenge in robot arm control for robotics researchers, though it is incremental as it builds on existing deep Q-learning methods.
The paper tackles the problem of training robot arm controllers by using 3D simulations with deep Q-learning to avoid the impracticality of real-world data collection, achieving a controller that can locate and grasp a cube from images without prior knowledge and showing preliminary transfer to a real robot without further training.
Recent trends in robot arm control have seen a shift towards end-to-end solutions, using deep reinforcement learning to learn a controller directly from raw sensor data, rather than relying on a hand-crafted, modular pipeline. However, the high dimensionality of the state space often means that it is impractical to generate sufficient training data with real-world experiments. As an alternative solution, we propose to learn a robot controller in simulation, with the potential of then transferring this to a real robot. Building upon the recent success of deep Q-networks, we present an approach which uses 3D simulations to train a 7-DOF robotic arm in a control task without any prior knowledge. The controller accepts images of the environment as its only input, and outputs motor actions for the task of locating and grasping a cube, over a range of initial configurations. To encourage efficient learning, a structured reward function is designed with intermediate rewards. We also present preliminary results in direct transfer of policies over to a real robot, without any further training.