Evaluating AI fairness in credit scoring with the BRIO tool
This work addresses fairness issues in credit scoring for financial institutions and regulators, but it is incremental as it applies an existing tool to a standard dataset.
The researchers tackled fairness evaluation in AI credit scoring systems by applying the BRIO tool to the UCI German Credit Dataset, quantifying fairness across demographic segments and identifying potential bias sources while combining results with revenue analysis.
We present a method for quantitative, in-depth analyses of fairness issues in AI systems with an application to credit scoring. To this aim we use BRIO, a tool for the evaluation of AI systems with respect to social unfairness and, more in general, ethically undesirable behaviours. It features a model-agnostic bias detection module, presented in \cite{DBLP:conf/beware/CoragliaDGGPPQ23}, to which a full-fledged unfairness risk evaluation module is added. As a case study, we focus on the context of credit scoring, analysing the UCI German Credit Dataset \cite{misc_statlog_(german_credit_data)_144}. We apply the BRIO fairness metrics to several, socially sensitive attributes featured in the German Credit Dataset, quantifying fairness across various demographic segments, with the aim of identifying potential sources of bias and discrimination in a credit scoring model. We conclude by combining our results with a revenue analysis.