Efficient Task-Oriented Dialogue Systems with Response Selection as an Auxiliary Task
This work addresses slow and non-diverse response generation in task-oriented dialogue systems, offering an incremental improvement for developers and users.
The paper tackled the inefficiency and lack of diversity in pre-trained language models for task-oriented dialogue systems by proposing two models with auxiliary tasks for response selection, achieving state-of-the-art results with combined scores of 107.5 and 108.3 on the MultiWOZ 2.1 dataset and outperforming a baseline with three times more parameters.
The adoption of pre-trained language models in task-oriented dialogue systems has resulted in significant enhancements of their text generation abilities. However, these architectures are slow to use because of the large number of trainable parameters and can sometimes fail to generate diverse responses. To address these limitations, we propose two models with auxiliary tasks for response selection - (1) distinguishing distractors from ground truth responses and (2) distinguishing synthetic responses from ground truth labels. They achieve state-of-the-art results on the MultiWOZ 2.1 dataset with combined scores of 107.5 and 108.3 and outperform a baseline with three times more parameters. We publish reproducible code and checkpoints and discuss the effects of applying auxiliary tasks to T5-based architectures.