Single-Sentence Reader: A Novel Approach for Addressing Answer Position Bias
This addresses robustness issues in MRC for AI researchers, but it is incremental as it focuses on a specific bias.
The paper tackled answer-position bias in Machine Reading Comprehension (MRC), where models rely on spurious correlations like answers being in the first sentence, and proposed a Single-Sentence Reader that nearly matches the performance of models trained on normal datasets in experiments with six models.
Machine Reading Comprehension (MRC) models tend to take advantage of spurious correlations (also known as dataset bias or annotation artifacts in the research community). Consequently, these models may perform the MRC task without fully comprehending the given context and question, which is undesirable since it may result in low robustness against distribution shift. The main focus of this paper is answer-position bias, where a significant percentage of training questions have answers located solely in the first sentence of the context. We propose a Single-Sentence Reader as a new approach for addressing answer position bias in MRC. Remarkably, in our experiments with six different models, our proposed Single-Sentence Readers trained on biased dataset achieve results that nearly match those of models trained on normal dataset, proving their effectiveness in addressing the answer position bias. Our study also discusses several challenges our Single-Sentence Readers encounter and proposes a potential solution.