Fairness without Sensitive Attributes via Knowledge Sharing
This addresses fairness in machine learning for applications where sensitive data is inaccessible, offering a novel approach that is incremental in method but practical for real-world privacy constraints.
The paper tackles the problem of improving model fairness when sensitive attributes are unavailable due to privacy concerns, by proposing a confidence-based hierarchical classifier called Reckoner that reduces bias through knowledge sharing between high- and low-confidence data subsets, achieving better accuracy and fairness than state-of-the-art methods on datasets like COMPAS and New Adult.
While model fairness improvement has been explored previously, existing methods invariably rely on adjusting explicit sensitive attribute values in order to improve model fairness in downstream tasks. However, we observe a trend in which sensitive demographic information becomes inaccessible as public concerns around data privacy grow. In this paper, we propose a confidence-based hierarchical classifier structure called "Reckoner" for reliable fair model learning under the assumption of missing sensitive attributes. We first present results showing that if the dataset contains biased labels or other hidden biases, classifiers significantly increase the bias gap across different demographic groups in the subset with higher prediction confidence. Inspired by these findings, we devised a dual-model system in which a version of the model initialised with a high-confidence data subset learns from a version of the model initialised with a low-confidence data subset, enabling it to avoid biased predictions. Our experimental results show that Reckoner consistently outperforms state-of-the-art baselines in COMPAS dataset and New Adult dataset, considering both accuracy and fairness metrics.