A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models
This work addresses task-oriented conversational modeling with unstructured knowledge access for dialog system challenges, but it is incremental as it builds on existing methods and a specific competition track.
The authors tackled the problem of building a knowledge-grounded dialog system for a specific challenge track by fine-tuning Transformer models on sub-tasks, resulting in performance and efficiency gains through techniques like entity-matching and pointer networks.
We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks in this challenge track. Specifically, we divided the task into four sub-tasks and fine-tuned several Transformer models on each of the sub-tasks. We made additional changes that yielded gains in both performance and efficiency, including the combination of the model with traditional entity-matching techniques, and the addition of a pointer network to the output layer of the language model.