Automated Item Neutralization for Non-Cognitive Scales: A Large Language Model Approach to Reducing Social-Desirability Bias
This addresses bias reduction in personality assessments for psychology and HR applications, but it is incremental as it builds on existing neutralization methods with AI assistance.
This study tackled social-desirability bias in personality assessments by using GPT-3 to rewrite items from the IPIP-BFM-50 scale, resulting in preserved reliability and a five-factor structure, with correlations to social desirability decreasing for some items but inconsistently.
This study evaluates item neutralization assisted by the large language model (LLM) to reduce social desirability bias in personality assessment. GPT-o3 was used to rewrite the International Personality Item Pool Big Five Measure (IPIP-BFM-50), and 203 participants completed either the original or neutralized form along with the Marlowe-Crowne Social Desirability Scale. The results showed preserved reliability and a five-factor structure, with gains in Conscientiousness and declines in Agreeableness and Openness. The correlations with social desirability decreased for several items, but inconsistently. Configural invariance held, though metric and scalar invariance failed. Findings support AI neutralization as a potential but imperfect bias-reduction method.