Reward Shaping for Human Learning via Inverse Reinforcement Learning
This work addresses the problem of accelerating human learning for specific tasks, offering a complementary approach to safety features in machine learning, though it is incremental as it builds on existing IRL methods.
The paper tackles the problem of slow human learning in challenging tasks by proposing reward shaping via inverse reinforcement learning as a learning aid, and shows with statistical significance that players receiving this aid approach desired policies more quickly than a control group in online game experiments.
Humans are spectacular reinforcement learners, constantly learning from and adjusting to experience and feedback. Unfortunately, this doesn't necessarily mean humans are fast learners. When tasks are challenging, learning can become unacceptably slow. Fortunately, humans do not have to learn tabula rasa, and learning speed can be greatly increased with learning aids. In this work we validate a new type of learning aid -- reward shaping for humans via inverse reinforcement learning (IRL). The goal of this aid is to increase the speed with which humans can learn good policies for specific tasks. Furthermore this approach compliments alternative machine learning techniques such as safety features that try to prevent individuals from making poor decisions. To achieve our results we first extend a well known IRL algorithm via kernel methods. Afterwards we conduct two human subjects experiments using an online game where players have limited time to learn a good policy. We show with statistical significance that players who receive our learning aid are able to approach desired policies more quickly than the control group.