DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization
This addresses the challenge of knowledge retrieval for conversational AI systems, but it is incremental as it builds on existing document-grounded dialogue methods.
The paper tackled the problem of identifying relevant knowledge in conversational systems grounded in long documents by introducing a model that uses document structure and dialogue context, achieving effectiveness on two datasets with generalization to unseen documents and long contexts.
Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge identification model that leverages the document structure to provide dialogue-contextualized passage encodings and better locate knowledge relevant to the conversation. An auxiliary loss captures the history of dialogue-document connections. We demonstrate the effectiveness of our model on two document-grounded conversational datasets and provide analyses showing generalization to unseen documents and long dialogue contexts.