Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models
This work addresses the everyday problem of multi-party conversations for dialogue systems, representing an incremental advancement by extending pairwise methods to group settings.
The paper tackles the problem of conversational agents in multi-party group settings, which current dialogue research largely ignores, by collecting and evaluating multi-party conversations using the LIGHT environment. The result shows that their new dataset, MultiLIGHT, brings significant improvements in this setting compared to existing pairwise-trained models and large language models with few-shot prompting.
Current dialogue research primarily studies pairwise (two-party) conversations, and does not address the everyday setting where more than two speakers converse together. In this work, we both collect and evaluate multi-party conversations to study this more general case. We use the LIGHT environment to construct grounded conversations, where each participant has an assigned character to role-play. We thus evaluate the ability of language models to act as one or more characters in such conversations. Models require two skills that pairwise-trained models appear to lack: (1) being able to decide when to talk; (2) producing coherent utterances grounded on multiple characters. We compare models trained on our new dataset to existing pairwise-trained dialogue models, as well as large language models with few-shot prompting. We find that our new dataset, MultiLIGHT, which we will publicly release, can help bring significant improvements in the group setting.