PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games
This work addresses the challenge of improving reasoning abilities for LLM-based agents in complex social scenarios, representing an incremental advancement in multi-agent communication.
The authors tackled the problem of evaluating and enhancing multi-agent reasoning in complex social settings like Murder Mystery Games by introducing a new dataset and a novel LLM-based agent framework. Their PLAYER* framework outperformed existing methods in reasoning accuracy, efficiency, and agent-human interaction.
We introduce WellPlay, a reasoning dataset for multi-agent conversational inference in Murder Mystery Games (MMGs). WellPlay comprises 1,482 inferential questions across 12 games, spanning objectives, reasoning, and relationship understanding, and establishes a systematic benchmark for evaluating agent reasoning abilities in complex social settings. Building on this foundation, we present PLAYER*, a novel framework for Large Language Model (LLM)-based agents in MMGs. MMGs pose unique challenges, including undefined state spaces, absent intermediate rewards, and the need for strategic reasoning through natural language. PLAYER* addresses these challenges with a sensor-based state representation and an information-driven strategy that optimises questioning and suspect pruning. Experiments show that PLAYER* outperforms existing methods in reasoning accuracy, efficiency, and agent-human interaction, advancing reasoning agents for complex social scenarios.