Boosting Reinforcement Learning and Planning with Demonstrations: A Survey
It addresses the problem of inefficient learning in complex environments for AI researchers and practitioners, but is incremental as it synthesizes existing methods.
This survey examines how demonstrations can improve reinforcement learning and planning by leveraging expert knowledge to overcome inefficiencies in trial-and-error learning, illustrated with examples from the ManiSkill robot learning benchmark.
Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments. The use of demonstrations, on the other hand, enables agents to benefit from expert knowledge rather than having to discover the best action to take through exploration. In this survey, we discuss the advantages of using demonstrations in sequential decision making, various ways to apply demonstrations in learning-based decision making paradigms (for example, reinforcement learning and planning in the learned models), and how to collect the demonstrations in various scenarios. Additionally, we exemplify a practical pipeline for generating and utilizing demonstrations in the recently proposed ManiSkill robot learning benchmark.