Deep Interpretable Models of Theory of Mind
This addresses the challenge of creating more effective and transparent AI for human interaction, though it appears incremental as it builds on existing agent-modeling approaches.
The paper tackles the problem of AI systems needing to both understand humans and be understandable by humans by developing an interpretable modular neural framework for modeling intentions, demonstrating on a Minecraft search and rescue task that interpretability can significantly increase predictive performance under certain conditions.
When developing AI systems that interact with humans, it is essential to design both a system that can understand humans, and a system that humans can understand. Most deep network based agent-modeling approaches are 1) not interpretable and 2) only model external behavior, ignoring internal mental states, which potentially limits their capability for assistance, interventions, discovering false beliefs, etc. To this end, we develop an interpretable modular neural framework for modeling the intentions of other observed entities. We demonstrate the efficacy of our approach with experiments on data from human participants on a search and rescue task in Minecraft, and show that incorporating interpretability can significantly increase predictive performance under the right conditions.