InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons
This addresses the challenge of making imitation learning accessible to non-experts, though it is incremental in improving user interaction for specific tasks.
The paper tackles the problem of enabling laypersons to effectively teach AI agents through imitation learning by proposing InterPReT, which allows interactive policy updates based on user instructions and demonstrations, resulting in more robust policies in a driving game user study without compromising usability.
Imitation learning has shown success in many tasks by learning from expert demonstrations. However, most existing work relies on large-scale demonstrations from technical professionals and close monitoring of the training process. These are challenging for a layperson when they want to teach the agent new skills. To lower the barrier of teaching AI agents, we propose Interactive Policy Restructuring and Training (InterPReT), which takes user instructions to continually update the policy structure and optimize its parameters to fit user demonstrations. This enables end-users to interactively give instructions and demonstrations, monitor the agent's performance, and review the agent's decision-making strategies. A user study (N=34) on teaching an AI agent to drive in a racing game confirms that our approach yields more robust policies without impairing system usability, compared to a generic imitation learning baseline, when a layperson is responsible for both giving demonstrations and determining when to stop. This shows that our method is more suitable for end-users without much technical background in machine learning to train a dependable policy