Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework
This work addresses fairness issues in high-stakes applications like legal systems, providing a practical refutation strategy for practitioners, but it is incremental as it builds on existing causal fairness research by focusing on unobserved confounding.
The paper tackles the problem of causal fairness in machine learning predictions when unobserved confounding exists, which can lead to unfair outcomes, by deriving bounds for fairness metrics and proposing a neural framework with worst-case guarantees, demonstrated through experiments including a real-world prison sentence prediction case study.
Fairness for machine learning predictions is widely required in practice for legal, ethical, and societal reasons. Existing work typically focuses on settings without unobserved confounding, even though unobserved confounding can lead to severe violations of causal fairness and, thus, unfair predictions. In this work, we analyze the sensitivity of causal fairness to unobserved confounding. Our contributions are three-fold. First, we derive bounds for causal fairness metrics under different sources of unobserved confounding. This enables practitioners to examine the sensitivity of their machine learning models to unobserved confounding in fairness-critical applications. Second, we propose a novel neural framework for learning fair predictions, which allows us to offer worst-case guarantees of the extent to which causal fairness can be violated due to unobserved confounding. Third, we demonstrate the effectiveness of our framework in a series of experiments, including a real-world case study about predicting prison sentences. To the best of our knowledge, ours is the first work to study causal fairness under unobserved confounding. To this end, our work is of direct practical value as a refutation strategy to ensure the fairness of predictions in high-stakes applications.