Efficient and Robust Question Answering from Minimal Context over Documents
This addresses scalability and robustness issues in QA systems for large document corpora, offering an incremental improvement over existing methods.
The paper tackles the inefficiency and vulnerability of neural QA models by proposing a sentence selector to identify minimal context, achieving up to 15x faster training and 13x faster inference with comparable or better accuracy on benchmarks like SQuAD and TriviaQA, while also improving robustness to adversarial inputs.
Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs.