The Confidence Trap: Gender Bias and Predictive Certainty in LLMs
This work addresses fairness concerns for LLM deployment in sensitive domains, though it is incremental as it focuses on evaluating existing models rather than proposing new methods.
This study examined how well LLMs' confidence scores align with gender bias in pronoun resolution tasks, finding that Gemma-2 showed the worst calibration among six state-of-the-art models. The researchers introduced a new metric, Gender-ECE, to measure gender disparities in these tasks.
The increased use of Large Language Models (LLMs) in sensitive domains leads to growing interest in how their confidence scores correspond to fairness and bias. This study examines the alignment between LLM-predicted confidence and human-annotated bias judgments. Focusing on gender bias, the research investigates probability confidence calibration in contexts involving gendered pronoun resolution. The goal is to evaluate if calibration metrics based on predicted confidence scores effectively capture fairness-related disparities in LLMs. The results show that, among the six state-of-the-art models, Gemma-2 demonstrates the worst calibration according to the gender bias benchmark. The primary contribution of this work is a fairness-aware evaluation of LLMs' confidence calibration, offering guidance for ethical deployment. In addition, we introduce a new calibration metric, Gender-ECE, designed to measure gender disparities in resolution tasks.