Making Efficient Use of Demonstrations to Solve Hard Exploration Problems
This addresses the problem of sample inefficiency in reinforcement learning for complex environments, though it appears incremental as it builds on existing demonstration-based methods.
The paper tackles hard exploration problems in partially observable environments with variable initial conditions by introducing R2D3, an agent that efficiently uses demonstrations, and shows it solves tasks where other state-of-the-art methods fail to see a single successful trajectory after tens of billions of exploration steps.
This paper introduces R2D3, an agent that makes efficient use of demonstrations to solve hard exploration problems in partially observable environments with highly variable initial conditions. We also introduce a suite of eight tasks that combine these three properties, and show that R2D3 can solve several of the tasks where other state of the art methods (both with and without demonstrations) fail to see even a single successful trajectory after tens of billions of steps of exploration.