Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference
This addresses a specific bias problem in NLI for researchers, but it is incremental as it focuses on an overlooked aspect of a known issue.
The paper tackles the reverse word-overlap bias in Natural Language Inference models, showing that they are highly biased towards non-entailment for low-overlap instances, with existing debiasing methods being ineffective, and finds that this bias does not originate from pre-training.
It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we focus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased towards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally ineffective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority examples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we observe that in contrast to the accepted assumption, eliminating minority examples does not affect the generalizability of debiasing methods with respect to the overlap bias.