Irrelevant Alternatives Bias Large Language Model Hiring Decisions
This research identifies a cognitive bias in LLMs that could impact automated hiring systems, highlighting a potential flaw in AI decision-making for recruitment.
The study investigated whether large language models (LLMs) exhibit the attraction effect, a human cognitive bias, in hiring decisions, finding consistent and significant evidence of this bias in GPT-3.5 and GPT-4, with GPT-4 showing greater variation and irrelevant attributes like gender amplifying the effect.
We investigate whether LLMs display a well-known human cognitive bias, the attraction effect, in hiring decisions. The attraction effect occurs when the presence of an inferior candidate makes a superior candidate more appealing, increasing the likelihood of the superior candidate being chosen over a non-dominated competitor. Our study finds consistent and significant evidence of the attraction effect in GPT-3.5 and GPT-4 when they assume the role of a recruiter. Irrelevant attributes of the decoy, such as its gender, further amplify the observed bias. GPT-4 exhibits greater bias variation than GPT-3.5. Our findings remain robust even when warnings against the decoy effect are included and the recruiter role definition is varied.