Can I Be of Further Assistance? Using Unstructured Knowledge Access to Improve Task-oriented Conversational Modeling
This addresses a practical limitation in conversational AI for users needing assistance beyond predefined APIs, though it appears incremental as it builds on existing pipeline architectures.
The paper tackles the problem of task-oriented dialogue systems failing to handle user requests beyond their API coverage by incorporating external unstructured knowledge sources. Their pipelined approach with novel data augmentation achieves state-of-the-art performance on the DSTC9 Track 1 benchmark.
Most prior work on task-oriented dialogue systems are restricted to limited coverage of domain APIs. However, users oftentimes have requests that are out of the scope of these APIs. This work focuses on responding to these beyond-API-coverage user turns by incorporating external, unstructured knowledge sources. Our approach works in a pipelined manner with knowledge-seeking turn detection, knowledge selection, and response generation in sequence. We introduce novel data augmentation methods for the first two steps and demonstrate that the use of information extracted from dialogue context improves the knowledge selection and end-to-end performances. Through experiments, we achieve state-of-the-art performance for both automatic and human evaluation metrics on the DSTC9 Track 1 benchmark dataset, validating the effectiveness of our contributions.