Machine Learning Fairness in Justice Systems: Base Rates, False Positives, and False Negatives
This addresses fairness issues in justice systems for impacted communities, but it is incremental as it builds on existing fairness discussions without introducing new methods.
The paper tackles the problem of achieving fairness in machine learning for justice systems, focusing on the trade-offs between false positives and false negatives across racial groups, and concludes that computational solutions are limited, requiring stakeholder engagement for practical implementation.
Machine learning best practice statements have proliferated, but there is a lack of consensus on what the standards should be. For fairness standards in particular, there is little guidance on how fairness might be achieved in practice. Specifically, fairness in errors (both false negatives and false positives) can pose a problem of how to set weights, how to make unavoidable tradeoffs, and how to judge models that present different kinds of errors across racial groups. This paper considers the consequences of having higher rates of false positives for one racial group and higher rates of false negatives for another racial group. The paper examines how different errors in justice settings can present problems for machine learning applications, the limits of computation for resolving tradeoffs, and how solutions might have to be crafted through courageous conversations with leadership, line workers, stakeholders, and impacted communities.