Training Neural Response Selection for Task-Oriented Dialogue Systems
This addresses the challenge of deploying retrieval-based models in task-oriented dialogue systems, which is incremental as it adapts existing pretraining techniques to a specific domain.
The paper tackled the problem of low-data regimes in task-oriented dialogue systems by proposing a pretraining and fine-tuning method for neural response selection, achieving effectiveness across six diverse application domains.
Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.