Revealing Hidden Bias in AI: Lessons from Large Language Models
It addresses bias in AI for recruitment, offering insights for fairness, but is incremental in evaluating existing models.
This study examined biases in candidate interview reports generated by large language models (LLMs) like Claude 3.5 Sonnet and GPT-4o, finding that anonymization reduces certain biases, with Llama 3.1 405B showing the lowest overall bias.
As large language models (LLMs) become integral to recruitment processes, concerns about AI-induced bias have intensified. This study examines biases in candidate interview reports generated by Claude 3.5 Sonnet, GPT-4o, Gemini 1.5, and Llama 3.1 405B, focusing on characteristics such as gender, race, and age. We evaluate the effectiveness of LLM-based anonymization in reducing these biases. Findings indicate that while anonymization reduces certain biases, particularly gender bias, the degree of effectiveness varies across models and bias types. Notably, Llama 3.1 405B exhibited the lowest overall bias. Moreover, our methodology of comparing anonymized and non-anonymized data reveals a novel approach to assessing inherent biases in LLMs beyond recruitment applications. This study underscores the importance of careful LLM selection and suggests best practices for minimizing bias in AI applications, promoting fairness and inclusivity.