A statistical approach to detect sensitive features in a group fairness setting
This addresses the issue of subjective and incomplete sensitive feature selection in fairness evaluations for machine learning systems, though it is incremental as it builds on existing statistical methods.
The paper tackles the problem of automatically detecting sensitive features in group fairness settings without needing a trained model, using the Hilbert-Schmidt independence criterion to measure statistical dependence between labels and candidate features, and finds that some literature-defined sensitive features do not necessarily lead to unfair outcomes.
The use of machine learning models in decision support systems with high societal impact raised concerns about unfair (disparate) results for different groups of people. When evaluating such unfair decisions, one generally relies on predefined groups that are determined by a set of features that are considered sensitive. However, such an approach is subjective and does not guarantee that these features are the only ones to be considered as sensitive nor that they entail unfair (disparate) outcomes. In this paper, we propose a preprocessing step to address the task of automatically recognizing sensitive features that does not require a trained model to verify unfair results. Our proposal is based on the Hilber-Schmidt independence criterion, which measures the statistical dependence of variable distributions. We hypothesize that if the dependence between the label vector and a candidate is high for a sensitive feature, then the information provided by this feature will entail disparate performance measures between groups. Our empirical results attest our hypothesis and show that several features considered as sensitive in the literature do not necessarily entail disparate (unfair) results.