Multi-trainer Interactive Reinforcement Learning System
This addresses the issue of unreliable human guidance in reinforcement learning for agents in reward-sparse environments, but it is incremental as it builds on existing interactive methods by adding multiple trainers.
The paper tackles the problem of unreliable human feedback in interactive reinforcement learning by proposing a multi-trainer system that aggregates binary feedback from multiple non-perfect trainers into a more reliable reward, showing that their aggregation method achieves the best accuracy compared to baselines and improves policy training in a grid-world experiment.
Interactive reinforcement learning can effectively facilitate the agent training via human feedback. However, such methods often require the human teacher to know what is the correct action that the agent should take. In other words, if the human teacher is not always reliable, then it will not be consistently able to guide the agent through its training. In this paper, we propose a more effective interactive reinforcement learning system by introducing multiple trainers, namely Multi-Trainer Interactive Reinforcement Learning (MTIRL), which could aggregate the binary feedback from multiple non-perfect trainers into a more reliable reward for an agent training in a reward-sparse environment. In particular, our trainer feedback aggregation experiments show that our aggregation method has the best accuracy when compared with the majority voting, the weighted voting, and the Bayesian method. Finally, we conduct a grid-world experiment to show that the policy trained by the MTIRL with the review model is closer to the optimal policy than that without a review model.