An experiment on the mechanisms of racial bias in ML-based credit scoring in Brazil
This addresses algorithmic fairness in credit scoring for Brazil's large Black population, highlighting the need for audit methods that don't require protected attributes.
The researchers investigated how location information introduces racial bias in a credit scoring model in Brazil without using protected attributes, demonstrating that classification parity measures based on geographic groups can reveal racial bias.
We dissect an experimental credit scoring model developed with real data and demonstrate - without access to protected attributes - how the use of location information introduces racial bias. We analyze the tree gradient boosting model with the aid of a game-theoretic inspired machine learning explainability technique, counterfactual experiments and Brazilian census data. By exposing algorithmic racial bias explaining the trained machine learning model inner mechanisms, this experiment comprises an interesting artifact to aid the endeavor of theoretical understanding of the emergence of racial bias in machine learning systems. Without access to individuals' racial categories, we show how classification parity measures using geographically defined groups could carry information about model racial bias. The experiment testifies to the need for methods and language that do not presuppose access to protected attributes when auditing ML models, the importance of considering regional specifics when addressing racial issues, and the central role of census data in the AI research community. To the best of our knowledge, this is the first documented case of algorithmic racial bias in ML-based credit scoring in Brazil, the country with the second largest Black population in the world.