Active Exploration for Robotic Manipulation
This addresses the problem of robotic manipulation for robotics researchers, presenting an incremental improvement in active exploration techniques.
The paper tackles the challenge of exploring environment dynamics in sparse-reward robotic manipulation by proposing a model-based active exploration method that uses an ensemble of probabilistic models and model predictive control to maximize reward and perform directed exploration. The method was evaluated in simulation and on a real robot for a ball pushing task, achieving efficient learning from scratch.
Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method, on a challenging ball pushing task on tilted tables, where the target ball position is not known to the agent a-priori. Our real-world robot experiment serves as a fundamental application of active exploration in model-based reinforcement learning of complex robotic manipulation tasks.