Revisiting the Open-Domain Question Answering Pipeline
This work addresses the problem of improving accuracy and flexibility in open-domain QA systems for researchers and practitioners, though it is incremental as it builds on existing pipeline components.
The paper tackles open-domain question answering by introducing Mindstone, a new multi-stage pipeline that achieves a new baseline performance on the Wikipedia/SQuAD dataset with EM=58.1 and F1=65.8, showing substantial gains over previous state-of-the-art methods.
Open-domain question answering (QA) is the tasl of identifying answers to natural questions from a large corpus of documents. The typical open-domain QA system starts with information retrieval to select a subset of documents from the corpus, which are then processed by a machine reader to select the answer spans. This paper describes Mindstone, an open-domain QA system that consists of a new multi-stage pipeline that employs a traditional BM25-based information retriever, RM3-based neural relevance feedback, neural ranker, and a machine reading comprehension stage. This paper establishes a new baseline for end-to-end performance on question answering for Wikipedia/SQuAD dataset (EM=58.1, F1=65.8), with substantial gains over the previous state of the art (Yang et al., 2019b). We also show how the new pipeline enables the use of low-resolution labels, and can be easily tuned to meet various timing requirements.