Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access Track in DSTC9
This work addresses the problem of expanding coverage for users of task-oriented dialogue systems beyond predefined APIs, though it is incremental as it builds on existing neural models.
The paper tackled the limitation of task-oriented dialogue systems being restricted to domain APIs by incorporating external unstructured knowledge sources, resulting in 105 entries from 24 teams and ensemble methods with large-scale pretrained models achieving high performance and better generalization.
Most prior work on task-oriented dialogue systems are restricted to a limited coverage of domain APIs, while users oftentimes have domain related requests that are not covered by the APIs. This challenge track aims to expand the coverage of task-oriented dialogue systems by incorporating external unstructured knowledge sources. We define three tasks: knowledge-seeking turn detection, knowledge selection, and knowledge-grounded response generation. We introduce the data sets and the neural baseline models for three tasks. The challenge track received a total of 105 entries from 24 participating teams. In the evaluation results, the ensemble methods with different large-scale pretrained language models achieved high performances with improved knowledge selection capability and better generalization into unseen data.