Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment
This work addresses the challenge of safely enhancing deterministic policies in public policy applications like criminal justice, where existing methods fail due to non-stochasticity, representing an incremental advance in safe policy learning.
The researchers tackled the problem of improving deterministic algorithmic pre-trial risk assessments in the US criminal justice system, developing a maximin robust optimization method that safely improved certain components by classifying arrestees as lower risk under various utility specifications, though it did not inform all components.
Algorithmic recommendations and decisions have become ubiquitous in today's society. Many of these data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. We examine a particular case of algorithmic pre-trial risk assessments in the US criminal justice system, which provide deterministic classification scores and recommendations to help judges make release decisions. Our goal is to analyze data from a unique field experiment on an algorithmic pre-trial risk assessment to investigate whether the scores and recommendations can be improved. Unfortunately, prior methods for policy learning are not applicable because they require existing policies to be stochastic. We develop a maximin robust optimization approach that partially identifies the expected utility of a policy, and then finds a policy that maximizes the worst-case expected utility. The resulting policy has a statistical safety property, limiting the probability of producing a worse policy than the existing one, under structural assumptions about the outcomes. Our analysis of data from the field experiment shows that we can safely improve certain components of the risk assessment instrument by classifying arrestees as lower risk under a wide range of utility specifications, though the analysis is not informative about several components of the instrument.