Relation Module for Non-answerable Prediction on Question Answering
This addresses the challenge of non-answerable prediction in question answering, which is incremental as it builds on existing models like BiDAF and BERT.
The paper tackles the problem of determining whether a question has an answer in a given context for machine reading comprehension, specifically on the SQuAD 2.0 dataset, by introducing a relation module that achieves a 1.8% F1 gain on BiDAF and 1.0% on BERT base models.
Machine reading comprehension(MRC) has attracted significant amounts of research attention recently, due to an increase of challenging reading comprehension datasets. In this paper, we aim to improve a MRC model's ability to determine whether a question has an answer in a given context (e.g. the recently proposed SQuAD 2.0 task). Our solution is a relation module that is adaptable to any MRC model. The relation module consists of both semantic extraction and relational information. We first extract high level semantics as objects from both question and context with multi-head self-attentive pooling. These semantic objects are then passed to a relation network, which generates relationship scores for each object pair in a sentence. These scores are used to determine whether a question is non-answerable. We test the relation module on the SQuAD 2.0 dataset using both BiDAF and BERT models as baseline readers. We obtain 1.8% gain of F1 on top of the BiDAF reader, and 1.0% on top of the BERT base model. These results show the effectiveness of our relation module on MRC