Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
This work addresses fairness evaluation for LLMs, which is critical for transparent AI, but it is incremental as it builds on existing fairness metrics by adding uncertainty awareness.
The authors tackled the problem of evaluating fairness in large language models (LLMs) by proposing an uncertainty-aware metric called UCerF, which revealed biases like Mistral-7B's high confidence in incorrect predictions that conventional metrics missed, and they introduced a new dataset with 31,756 samples for benchmarking.
The recent rapid adoption of large language models (LLMs) highlights the critical need for benchmarking their fairness. Conventional fairness metrics, which focus on discrete accuracy-based evaluations (i.e., prediction correctness), fail to capture the implicit impact of model uncertainty (e.g., higher model confidence about one group over another despite similar accuracy). To address this limitation, we propose an uncertainty-aware fairness metric, UCerF, to enable a fine-grained evaluation of model fairness that is more reflective of the internal bias in model decisions compared to conventional fairness measures. Furthermore, observing data size, diversity, and clarity issues in current datasets, we introduce a new gender-occupation fairness evaluation dataset with 31,756 samples for co-reference resolution, offering a more diverse and suitable dataset for evaluating modern LLMs. We establish a benchmark, using our metric and dataset, and apply it to evaluate the behavior of ten open-source LLMs. For example, Mistral-7B exhibits suboptimal fairness due to high confidence in incorrect predictions, a detail overlooked by Equalized Odds but captured by UCerF. Overall, our proposed LLM benchmark, which evaluates fairness with uncertainty awareness, paves the way for developing more transparent and accountable AI systems.