Inferring the Goals of Communicating Agents from Actions and Instructions
This work addresses the challenge of improving goal inference for cooperative AI agents, but it is incremental as it builds on existing Bayesian and language models in a specific domain.
The paper tackled the problem of modeling how humans infer shared goals from both actions and verbal instructions in cooperative settings, by introducing a Bayesian inverse planning model that uses GPT-3 for instruction likelihood. The result showed that the model's goal inferences closely correlated with human judgments (R = 0.96) and that instructions led to faster and less uncertain inference compared to actions alone.
When humans cooperate, they frequently coordinate their activity through both verbal communication and non-verbal actions, using this information to infer a shared goal and plan. How can we model this inferential ability? In this paper, we introduce a model of a cooperative team where one agent, the principal, may communicate natural language instructions about their shared plan to another agent, the assistant, using GPT-3 as a likelihood function for instruction utterances. We then show how a third person observer can infer the team's goal via multi-modal Bayesian inverse planning from actions and instructions, computing the posterior distribution over goals under the assumption that agents will act and communicate rationally to achieve them. We evaluate this approach by comparing it with human goal inferences in a multi-agent gridworld, finding that our model's inferences closely correlate with human judgments (R = 0.96). When compared to inference from actions alone, we also find that instructions lead to more rapid and less uncertain goal inference, highlighting the importance of verbal communication for cooperative agents.