Accelerating the Learning of TAMER with Counterfactual Explanations
This addresses the frustrating experience for users in interactive learning settings, but it is incremental as it builds on an existing framework.
The paper tackles the slow learning speed in human-in-the-loop reinforcement learning by extending the TAMER framework with counterfactual explanations, showing experimentally that this improves learning speed.
The capability to interactively learn from human feedback would enable agents in new settings. For example, even novice users could train service robots in new tasks naturally and interactively. Human-in-the-loop Reinforcement Learning (HRL) combines human feedback and Reinforcement Learning (RL) techniques. State-of-the-art interactive learning techniques suffer from slow learning speed, thus leading to a frustrating experience for the human. We approach this problem by extending the HRL framework TAMER for evaluative feedback with the possibility to enhance human feedback with two different types of counterfactual explanations (action and state based). We experimentally show that our extensions improve the speed of learning.