Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System
This work addresses the problem of building efficient and accurate task-oriented dialogue systems for applications like virtual assistants, though it is incremental as it builds on pre-trained language models.
The authors tackled the problem of error accumulation and high data annotation overhead in task-oriented dialogue systems by introducing PPTOD, a unified plug-and-play model with multi-task pre-training, which achieved state-of-the-art results on three benchmark tasks in both high-resource and low-resource scenarios, with human evaluations showing more factually correct and semantically coherent responses.
Pre-trained language models have been recently shown to benefit task-oriented dialogue (TOD) systems. Despite their success, existing methods often formulate this task as a cascaded generation problem which can lead to error accumulation across different sub-tasks and greater data annotation overhead. In this study, we present PPTOD, a unified plug-and-play model for task-oriented dialogue. In addition, we introduce a new dialogue multi-task pre-training strategy that allows the model to learn the primary TOD task completion skills from heterogeneous dialog corpora. We extensively test our model on three benchmark TOD tasks, including end-to-end dialogue modelling, dialogue state tracking, and intent classification. Experimental results show that PPTOD achieves new state of the art on all evaluated tasks in both high-resource and low-resource scenarios. Furthermore, comparisons against previous SOTA methods show that the responses generated by PPTOD are more factually correct and semantically coherent as judged by human annotators.