Recentering Validity Considerations through Early-Stage Deliberations Around AI and Policy Design
It tackles the problem of retrospective policy-AI alignment for stakeholders in domains like healthcare and criminal justice, but is incremental as it builds on existing calls for scrutiny.
The paper addresses the challenge of integrating validity considerations in AI tools with policy design, advocating for early-stage multi-stakeholder deliberations to bridge this gap.
AI-based decision-making tools are rapidly spreading across a range of real-world, complex domains like healthcare, criminal justice, and child welfare. A growing body of research has called for increased scrutiny around the validity of AI system designs. However, in real-world settings, it is often not possible to fully address questions around the validity of an AI tool without also considering the design of associated organizational and public policies. Yet, considerations around how an AI tool may interface with policy are often only discussed retrospectively, after the tool is designed or deployed. In this short position paper, we discuss opportunities to promote multi-stakeholder deliberations around the design of AI-based technologies and associated policies, at the earliest stages of a new project.