Explaining Decentralized Multi-Agent Reinforcement Learning Policies
This addresses the need for interpretability in decentralized MARL for users like researchers or practitioners, though it appears incremental as it extends explanation methods from centralized to decentralized settings.
The paper tackles the problem of explaining decentralized multi-agent reinforcement learning policies, which existing methods fail to address due to uncertainty and nondeterminism. It proposes methods for policy summarizations and query-based explanations, showing in user studies that these significantly improve question-answering performance and subjective ratings like understanding and satisfaction.
Multi-Agent Reinforcement Learning (MARL) has gained significant interest in recent years, enabling sequential decision-making across multiple agents in various domains. However, most existing explanation methods focus on centralized MARL, failing to address the uncertainty and nondeterminism inherent in decentralized settings. We propose methods to generate policy summarizations that capture task ordering and agent cooperation in decentralized MARL policies, along with query-based explanations for When, Why Not, and What types of user queries about specific agent behaviors. We evaluate our approach across four MARL domains and two decentralized MARL algorithms, demonstrating its generalizability and computational efficiency. User studies show that our summarizations and explanations significantly improve user question-answering performance and enhance subjective ratings on metrics such as understanding and satisfaction.