The Limits of Computation in Solving Equity Trade-Offs in Machine Learning and Justice System Risk Assessment
This paper addresses a critical problem for policymakers and practitioners in the justice system by demonstrating the limitations of computational approaches in achieving racial equity in risk assessment tools, suggesting that values and mission must guide decisions where computation falls short.
This paper investigates the inherent difficulties in computationally resolving racial equity trade-offs within machine learning models, especially in justice system risk assessments. It highlights how different notions of racial equity lead to problematic score distributions and outcome rates across racial groups, which computation alone cannot resolve.
This paper explores how different ideas of racial equity in machine learning, in justice settings in particular, can present trade-offs that are difficult to solve computationally. Machine learning is often used in justice settings to create risk assessments, which are used to determine interventions, resources, and punitive actions. Overall aspects and performance of these machine learning-based tools, such as distributions of scores, outcome rates by levels, and the frequency of false positives and true positives, can be problematic when examined by racial group. Models that produce different distributions of scores or produce a different relationship between level and outcome are problematic when those scores and levels are directly linked to the restriction of individual liberty and to the broader context of racial inequity. While computation can help highlight these aspects, data and computation are unlikely to solve them. This paper explores where values and mission might have to fill the spaces computation leaves.