Reading Comprehension as Natural Language Inference: A Semantic Analysis
This addresses how to improve question answering systems for NLP researchers, though it appears incremental as it applies an existing method to reformatted data.
The paper investigated whether transforming reading comprehension questions into natural language inference format improves performance, finding that RoBERTa performed better on certain question categories when presented in entailment form versus traditional question-answer format.
In the recent past, Natural language Inference (NLI) has gained significant attention, particularly given its promise for downstream NLP tasks. However, its true impact is limited and has not been well studied. Therefore, in this paper, we explore the utility of NLI for one of the most prominent downstream tasks, viz. Question Answering (QA). We transform the one of the largest available MRC dataset (RACE) to an NLI form, and compare the performances of a state-of-the-art model (RoBERTa) on both these forms. We propose new characterizations of questions, and evaluate the performance of QA and NLI models on these categories. We highlight clear categories for which the model is able to perform better when the data is presented in a coherent entailment form, and a structured question-answer concatenation form, respectively.