Analyzing and Mitigating Negation Artifacts using Data Augmentation for Improving ELECTRA-Small Model Accuracy
This addresses a dataset artifact issue for natural language inference tasks, but it is incremental as it focuses on a specific model and data augmentation technique.
The paper tackled the problem of pre-trained models relying on spurious correlations rather than understanding negation in natural language inference, and found that targeted data augmentation with contrast sets and adversarial examples improved the ELECTRA-small model's accuracy on negation-containing examples without harming overall performance.
Pre-trained models for natural language inference (NLI) often achieve high performance on benchmark datasets by using spurious correlations, or dataset artifacts, rather than understanding language touches such as negation. In this project, we investigate the performance of an ELECTRA-small model fine-tuned on the Stanford Natural Language Inference (SNLI) dataset, focusing on its handling of negation. Through analysis, we identify that the model struggles with correctly classifying examples containing negation. To address this, we augment the training data with contrast sets and adversarial examples emphasizing negation. Our results demonstrate that this targeted data augmentation improves the model's accuracy on negation-containing examples without adversely affecting overall performance, therefore mitigating the identified dataset artifact.