A Task-oriented Dialog Model with Task-progressive and Policy-aware Pre-training
This work addresses the challenge of improving task-oriented dialog systems for users by enhancing pre-training efficiency and performance, though it appears incremental as it builds on existing PCMs with specific modifications.
The paper tackles the problem of pre-trained conversation models insufficiently capturing sequential task logic and dialog policy in task-oriented dialog, proposing a task-progressive model with policy-aware pre-training that achieves better results on MultiWOZ and In-Car benchmarks with only 18% parameters and 25% pre-training data compared to the previous SOTA.
Pre-trained conversation models (PCMs) have achieved promising progress in recent years. However, existing PCMs for Task-oriented dialog (TOD) are insufficient for capturing the sequential nature of the TOD-related tasks, as well as for learning dialog policy information. To alleviate these problems, this paper proposes a task-progressive PCM with two policy-aware pre-training tasks. The model is pre-trained through three stages where TOD-related tasks are progressively employed according to the task logic of the TOD system. A global policy consistency task is designed to capture the multi-turn dialog policy sequential relation, and an act-based contrastive learning task is designed to capture similarities among samples with the same dialog policy. Our model achieves better results on both MultiWOZ and In-Car end-to-end dialog modeling benchmarks with only 18\% parameters and 25\% pre-training data compared to the previous state-of-the-art PCM, GALAXY.