Acquiescence Bias in Large Language Models
This identifies a behavioral quirk in LLMs that could affect their reliability in applications like surveys or decision-making, though it is incremental as it builds on known human bias research.
The study investigated whether large language models (LLMs) exhibit acquiescence bias, a human tendency to agree, and found that LLMs instead show a bias towards answering 'no' across different models, tasks, and languages.
Acquiescence bias, i.e. the tendency of humans to agree with statements in surveys, independent of their actual beliefs, is well researched and documented. Since Large Language Models (LLMs) have been shown to be very influenceable by relatively small changes in input and are trained on human-generated data, it is reasonable to assume that they could show a similar tendency. We present a study investigating the presence of acquiescence bias in LLMs across different models, tasks, and languages (English, German, and Polish). Our results indicate that, contrary to humans, LLMs display a bias towards answering no, regardless of whether it indicates agreement or disagreement.