Learning Shaping Strategies in Human-in-the-loop Interactive Reinforcement Learning
This work addresses the challenge of robust learning in human-robot collaboration by adaptively integrating human knowledge, though it is incremental as it builds on prior shaping methods.
The paper tackles the problem of selecting the most suitable human guidance method in interactive reinforcement learning by proposing an adaptive shaping algorithm that learns the optimal shaping strategy online, achieving improved learning performance in both simulated and real human studies.
Providing reinforcement learning agents with informationally rich human knowledge can dramatically improve various aspects of learning. Prior work has developed different kinds of shaping methods that enable agents to learn efficiently in complex environments. All these methods, however, tailor human guidance to agents in specialized shaping procedures, thus embodying various characteristics and advantages in different domains. In this paper, we investigate the interplay between different shaping methods for more robust learning performance. We propose an adaptive shaping algorithm which is capable of learning the most suitable shaping method in an on-line manner. Results in two classic domains verify its effectiveness from both simulated and real human studies, shedding some light on the role and impact of human factors in human-robot collaborative learning.