A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection
This work addresses efficiency and accuracy in QA systems, offering a faster alternative to large neural models, but it is incremental as it builds on existing ranking and encoding techniques.
The paper tackled the problem of answer sentence selection in QA systems by proposing a method that leverages document structure and word-relatedness encoding to achieve competitive results with high efficiency, reducing training time from ~18 minutes to 9.5 seconds on the WikiQA dataset.
An essential task of most Question Answering (QA) systems is to re-rank the set of answer candidates, i.e., Answer Sentence Selection (A2S). These candidates are typically sentences either extracted from one or more documents preserving their natural order or retrieved by a search engine. Most state-of-the-art approaches to the task use huge neural models, such as BERT, or complex attentive architectures. In this paper, we argue that by exploiting the intrinsic structure of the original rank together with an effective word-relatedness encoder, we can achieve competitive results with respect to the state of the art while retaining high efficiency. Our model takes 9.5 seconds to train on the WikiQA dataset, i.e., very fast in comparison with the $\sim 18$ minutes required by a standard BERT-base fine-tuning.