Open-Retrieval Conversational Machine Reading
This addresses the limitation of prior works that assume rule texts are provided, making it more applicable to real-world scenarios like benefit qualification systems, though it is incremental by extending to multi-passage retrieval.
The paper tackles the problem of conversational machine reading in an open-retrieval setting, where systems must retrieve relevant rule texts from a collection to answer high-level questions or ask follow-up clarifications, and achieves state-of-the-art performance on the OR-ShARC dataset, outperforming existing models by a large margin.
In conversational machine reading, systems need to interpret natural language rules, answer high-level questions such as "May I qualify for VA health care benefits?", and ask follow-up clarification questions whose answer is necessary to answer the original question. However, existing works assume the rule text is provided for each user question, which neglects the essential retrieval step in real scenarios. In this work, we propose and investigate an open-retrieval setting of conversational machine reading. In the open-retrieval setting, the relevant rule texts are unknown so that a system needs to retrieve question-relevant evidence from a collection of rule texts, and answer users' high-level questions according to multiple retrieved rule texts in a conversational manner. We propose MUDERN, a Multi-passage Discourse-aware Entailment Reasoning Network which extracts conditions in the rule texts through discourse segmentation, conducts multi-passage entailment reasoning to answer user questions directly, or asks clarification follow-up questions to inquiry more information. On our created OR-ShARC dataset, MUDERN achieves the state-of-the-art performance, outperforming existing single-passage conversational machine reading models as well as a new multi-passage conversational machine reading baseline by a large margin. In addition, we conduct in-depth analyses to provide new insights into this new setting and our model.