Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference
This work highlights a problem for researchers and practitioners in NLP by showing that LLM-generated data can inherit biases, potentially affecting model training and evaluation, though it is incremental as it builds on known issues with human-crowdsourced artifacts.
The study tested whether using large language models (LLMs) to generate hypotheses for Natural Language Inference (NLI) datasets introduces annotation artifacts, similar to those found in human-crowdsourced data, and found that BERT-based classifiers achieved 86-96% accuracy on LLM-elicited datasets, indicating persistent biases.
We test whether replacing crowdsource workers with LLMs to write Natural Language Inference (NLI) hypotheses similarly results in annotation artifacts. We recreate a portion of the Stanford NLI corpus using GPT-4, Llama-2 and Mistral 7b, and train hypothesis-only classifiers to determine whether LLM-elicited hypotheses contain annotation artifacts. On our LLM-elicited NLI datasets, BERT-based hypothesis-only classifiers achieve between 86-96% accuracy, indicating these datasets contain hypothesis-only artifacts. We also find frequent "give-aways" in LLM-generated hypotheses, e.g. the phrase "swimming in a pool" appears in more than 10,000 contradictions generated by GPT-4. Our analysis provides empirical evidence that well-attested biases in NLI can persist in LLM-generated data.