A fuzzy-rough uncertainty measure to discover bias encoded explicitly or implicitly in features of structured pattern classification datasets
This work addresses bias detection in structured datasets for pattern recognition, which is important for academia, legislators, and enterprises, but it is incremental as it builds on previous research by exploring implicit bias and sensitivity.
The paper tackles the problem of measuring bias in tabular data for pattern classification by extending a fuzzy-rough uncertainty measure to detect bias encoded implicitly in non-protected features through correlation analysis, resulting in four scenarios for domain experts to evaluate and a sensitivity analysis to optimize fuzzy operators and distance functions.
The need to measure bias encoded in tabular data that are used to solve pattern recognition problems is widely recognized by academia, legislators and enterprises alike. In previous work, we proposed a bias quantification measure, called fuzzy-rough uncer-tainty, which relies on the fuzzy-rough set theory. The intuition dictates that protected features should not change the fuzzy-rough boundary regions of a decision class significantly. The extent to which this happens is a proxy for bias expressed as uncertainty in adecision-making context. Our measure's main advantage is that it does not depend on any machine learning prediction model but adistance function. In this paper, we extend our study by exploring the existence of bias encoded implicitly in non-protected featuresas defined by the correlation between protected and unprotected attributes. This analysis leads to four scenarios that domain experts should evaluate before deciding how to tackle bias. In addition, we conduct a sensitivity analysis to determine the fuzzy operatorsand distance function that best capture change in the boundary regions.