Auditing ML Models for Individual Bias and Unfairness
This work addresses fairness issues in ML models, particularly for individuals affected by biased predictions in domains like criminal justice, though it appears incremental as it builds on existing auditing frameworks with new statistical tools.
The authors tackled the problem of auditing machine learning models for individual bias and unfairness by formalizing it as an optimization problem and developing inferential tools for asymptotic confidence intervals and hypothesis tests, applying these tools to reveal gender and racial biases in the COMPAS recidivism prediction instrument.
We consider the task of auditing ML models for individual bias/unfairness. We formalize the task in an optimization problem and develop a suite of inferential tools for the optimal value. Our tools permit us to obtain asymptotic confidence intervals and hypothesis tests that cover the target/control the Type I error rate exactly. To demonstrate the utility of our tools, we use them to reveal the gender and racial biases in Northpointe's COMPAS recidivism prediction instrument.