Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction
This work improves the ability of question-answering systems to handle ambiguous questions, which is a common problem for general users in open-domain settings.
The paper addresses ambiguous questions in open-domain QA by proposing a model that aggregates evidence to predict single or multiple answers. It also introduces a round-trip prediction approach to iteratively generate and filter interpretations. The model achieves state-of-the-art performance on the AmbigQA dataset and competitive results on NQ-Open and TriviaQA.
In open-domain question answering, questions are highly likely to be ambiguous because users may not know the scope of relevant topics when formulating them. Therefore, a system needs to find possible interpretations of the question, and predict one or multiple plausible answers. When multiple plausible answers are found, the system should rewrite the question for each answer to resolve the ambiguity. In this paper, we present a model that aggregates and combines evidence from multiple passages to adaptively predict a single answer or a set of question-answer pairs for ambiguous questions. In addition, we propose a novel round-trip prediction approach to iteratively generate additional interpretations that our model fails to find in the first pass, and then verify and filter out the incorrect question-answer pairs to arrive at the final disambiguated output. Our model, named Refuel, achieves a new state-of-the-art performance on the AmbigQA dataset, and shows competitive performance on NQ-Open and TriviaQA. The proposed round-trip prediction is a model-agnostic general approach for answering ambiguous open-domain questions, which improves our Refuel as well as several baseline models. We release source code for our models and experiments at https://github.com/amzn/refuel-open-domain-qa.