Dynamic Teaching in Sequential Decision Making Environments
This work addresses teaching in sequential settings for AI/ML systems, but it appears incremental as it extends existing supervised protocols like Teaching Dimension to handle noise and sequences in MDPs.
The paper tackles the problem of teaching a model through demonstration in sequential decision-making environments by focusing on the teacher as a dynamic decision maker, providing theoretical bounds on learnability and a practical algorithm for dynamic teaching.
We describe theoretical bounds and a practical algorithm for teaching a model by demonstration in a sequential decision making environment. Unlike previous efforts that have optimized learners that watch a teacher demonstrate a static policy, we focus on the teacher as a decision maker who can dynamically choose different policies to teach different parts of the environment. We develop several teaching frameworks based on previously defined supervised protocols, such as Teaching Dimension, extending them to handle noise and sequences of inputs encountered in an MDP.We provide theoretical bounds on the learnability of several important model classes in this setting and suggest a practical algorithm for dynamic teaching.