Explainable Action Advising for Multi-Agent Reinforcement Learning
This addresses the challenge of sample efficiency and generalization in multi-agent reinforcement learning, offering an incremental improvement over existing action advising techniques.
The paper tackled the problem of knowledge transfer in reinforcement learning by introducing Explainable Action Advising, where a teacher provides action advice with explanations to improve student learning; results showed improved policy returns and convergence rates compared to state-of-the-art methods in single-agent and multi-agent scenarios.
Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample efficiency and policy performance. Such advice is commonly given in the form of state-action pairs. However, it makes it difficult for the student to reason with and apply to novel states. We introduce Explainable Action Advising, in which the teacher provides action advice as well as associated explanations indicating why the action was chosen. This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal. We empirically show that our framework is effective in both single-agent and multi-agent scenarios, yielding improved policy returns and convergence rates when compared to state-of-the-art methods