Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems
This addresses a bottleneck in knowledge retrieval for dialog systems, offering a novel method to handle correlated knowledge pieces, though it is incremental in improving existing retrieval frameworks.
The paper tackled the problem of retrieving multiple relevant and correlated knowledge pieces in knowledge-grounded dialog systems, where current models assume conditional independence, and proposed Entriever, an energy-based retriever that substantially outperforms a strong cross-encoder baseline and significantly improves end-to-end dialog system performance.
A retriever, which retrieves relevant knowledge pieces from a knowledge base given a context, is an important component in many natural language processing (NLP) tasks. Retrievers have been introduced in knowledge-grounded dialog systems to improve knowledge acquisition. In knowledge-grounded dialog systems, when conditioning on a given context, there may be multiple relevant and correlated knowledge pieces. However, knowledge pieces are usually assumed to be conditionally independent in current retriever models. To address this issue, we propose Entriever, an energy-based retriever. Entriever directly models the candidate retrieval results as a whole instead of modeling the knowledge pieces separately, with the relevance score defined by an energy function. We explore various architectures of energy functions and different training methods for Entriever, and show that Entriever substantially outperforms the strong cross-encoder baseline in knowledge retrieval tasks. Furthermore, we show that in semi-supervised training of knowledge-grounded dialog systems, Entriever enables effective scoring of retrieved knowledge pieces and significantly improves end-to-end performance of dialog systems.