Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems
This provides a more realistic dataset for building task-oriented dialogue systems in customer support, addressing multi-step procedures and policies, though it is incremental in expanding dataset scope.
The paper tackles the limitation of existing goal-oriented dialogue datasets by introducing the Action-Based Conversations Dataset (ABCD), a corpus with over 10K human-to-human dialogues for customer service, and shows that while advanced models outperform simpler ones, a 50.8% accuracy gap remains to human-level performance.
Existing goal-oriented dialogue datasets focus mainly on identifying slots and values. However, customer support interactions in reality often involve agents following multi-step procedures derived from explicitly-defined company policies as well. To study customer service dialogue systems in more realistic settings, we introduce the Action-Based Conversations Dataset (ABCD), a fully-labeled dataset with over 10K human-to-human dialogues containing 55 distinct user intents requiring unique sequences of actions constrained by policies to achieve task success. We propose two additional dialog tasks, Action State Tracking and Cascading Dialogue Success, and establish a series of baselines involving large-scale, pre-trained language models on this dataset. Empirical results demonstrate that while more sophisticated networks outperform simpler models, a considerable gap (50.8% absolute accuracy) still exists to reach human-level performance on ABCD.