Relation-Guided Pre-Training for Open-Domain Question Answering
This addresses a specific bottleneck in open-domain QA for users needing better performance on underrepresented relations, but it is incremental as it builds on existing models like DPR.
The paper tackles the problem of imbalanced relation types in open-domain question answering datasets, which hurts generalization on questions with long-tail relations, by proposing a Relation-Guided Pre-Training framework that improves Exact Match accuracy by 2.2% to 6.3% on benchmarks like Natural Questions and TriviaQA.
Answering complex open-domain questions requires understanding the latent relations between involving entities. However, we found that the existing QA datasets are extremely imbalanced in some types of relations, which hurts the generalization performance over questions with long-tail relations. To remedy this problem, in this paper, we propose a Relation-Guided Pre-Training (RGPT-QA) framework. We first generate a relational QA dataset covering a wide range of relations from both the Wikidata triplets and Wikipedia hyperlinks. We then pre-train a QA model to infer the latent relations from the question, and then conduct extractive QA to get the target answer entity. We demonstrate that by pretraining with propoed RGPT-QA techique, the popular open-domain QA model, Dense Passage Retriever (DPR), achieves 2.2%, 2.4%, and 6.3% absolute improvement in Exact Match accuracy on Natural Questions, TriviaQA, and WebQuestions. Particularly, we show that RGPT-QA improves significantly on questions with long-tail relations