Curiosity-Driven Multi-Criteria Hindsight Experience Replay
This work addresses a longstanding problem in reinforcement learning for robotics and simulation tasks, representing an incremental improvement over existing methods.
The paper tackled the challenge of sparse rewards in reinforcement learning by combining hindsight experience replay, curiosity-driven exploration, and curriculum learning to solve a complex block stacking task, achieving the first successful stacking of more than two blocks using only sparse rewards and no human demonstrations.
Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the prediction error of a learned dynamics model as an intrinsic reward has been shown to be effective for exploring a number of sparse-reward environments. We present a method that combines hindsight with curiosity-driven exploration and curriculum learning in order to solve the challenging sparse-reward block stacking task. We are the first to stack more than two blocks using only sparse reward without human demonstrations.