Persistent Rule-based Interactive Reinforcement Learning
This work is significant for researchers and practitioners in interactive reinforcement learning by enabling more efficient and less burdensome human-agent collaboration through persistent, generalizable advice.
This paper addresses the limitation of real-time, single-use advice in interactive reinforcement learning by proposing a method to retain and reuse provided knowledge. Their approach, persistent rule-based interactive reinforcement learning, substantially improves agent performance and reduces trainer interactions compared to existing methods, while rule-based advice achieves similar performance impact to state-based advice with fewer interactions.
Interactive reinforcement learning has allowed speeding up the learning process in autonomous agents by including a human trainer providing extra information to the agent in real-time. Current interactive reinforcement learning research has been limited to real-time interactions that offer relevant user advice to the current state only. Additionally, the information provided by each interaction is not retained and instead discarded by the agent after a single-use. In this work, we propose a persistent rule-based interactive reinforcement learning approach, i.e., a method for retaining and reusing provided knowledge, allowing trainers to give general advice relevant to more than just the current state. Our experimental results show persistent advice substantially improves the performance of the agent while reducing the number of interactions required for the trainer. Moreover, rule-based advice shows similar performance impact as state-based advice, but with a substantially reduced interaction count.