Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users
This work addresses the need for dialogue systems to handle collaborative decision-making among multiple users, though it is incremental as it builds on existing datasets and methods.
The authors tackled the problem of developing dialogue systems for conversations with multiple users by releasing the Multi-User MultiWOZ dataset, which includes dialogues among two users and one agent, and demonstrated that using predicted rewrites of multi-user chats into single-user queries improves dialogue state tracking by 15% in accuracy without modifying existing systems.
While most task-oriented dialogues assume conversations between the agent and one user at a time, dialogue systems are increasingly expected to communicate with multiple users simultaneously who make decisions collaboratively. To facilitate development of such systems, we release the Multi-User MultiWOZ dataset: task-oriented dialogues among two users and one agent. To collect this dataset, each user utterance from MultiWOZ 2.2 was replaced with a small chat between two users that is semantically and pragmatically consistent with the original user utterance, thus resulting in the same dialogue state and system response. These dialogues reflect interesting dynamics of collaborative decision-making in task-oriented scenarios, e.g., social chatter and deliberation. Supported by this data, we propose the novel task of multi-user contextual query rewriting: to rewrite a task-oriented chat between two users as a concise task-oriented query that retains only task-relevant information and that is directly consumable by the dialogue system. We demonstrate that in multi-user dialogues, using predicted rewrites substantially improves dialogue state tracking without modifying existing dialogue systems that are trained for single-user dialogues. Further, this method surpasses training a medium-sized model directly on multi-user dialogues and generalizes to unseen domains.