GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
This addresses the challenge of incomplete information in human-AI cooperation for tasks like gaming and household simulation, offering a novel framework for proactive verbal communication.
The paper tackles the problem of enabling embodied AI assistants to proactively communicate with humans for better cooperation by proposing Goal-Oriented Mental Alignment (GOMA), which formulates communication as a planning problem to minimize mental misalignment, resulting in improved cooperation performance and human perception in Overcooked and VirtualHome environments.
Verbal communication plays a crucial role in human cooperation, particularly when the partners only have incomplete information about the task, environment, and each other's mental state. In this paper, we propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA). GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the parts of agents' mental states that are relevant to the goals. This approach enables an embodied assistant to reason about when and how to proactively initialize communication with humans verbally using natural language to help achieve better cooperation. We evaluate our approach against strong baselines in two challenging environments, Overcooked (a multiplayer game) and VirtualHome (a household simulator). Our experimental results demonstrate that large language models struggle with generating meaningful communication that is grounded in the social and physical context. In contrast, our approach can successfully generate concise verbal communication for the embodied assistant to effectively boost the performance of the cooperation as well as human users' perception of the assistant.