Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
This addresses fairness gaps in public sector algorithms affecting vulnerable populations, but is incremental as it identifies existing problems rather than solving them.
The study interviewed 27 public sector ML practitioners across 5 OECD countries to understand challenges in incorporating fairness and accountability into high-stakes decision-making systems like taxation and justice, revealing a disconnect between organizational realities and current research that creates practical implementation gaps.
Calls for heightened consideration of fairness and accountability in algorithmically-informed public decisions---like taxation, justice, and child protection---are now commonplace. How might designers support such human values? We interviewed 27 public sector machine learning practitioners across 5 OECD countries regarding challenges understanding and imbuing public values into their work. The results suggest a disconnect between organisational and institutional realities, constraints and needs, and those addressed by current research into usable, transparent and 'discrimination-aware' machine learning---absences likely to undermine practical initiatives unless addressed. We see design opportunities in this disconnect, such as in supporting the tracking of concept drift in secondary data sources, and in building usable transparency tools to identify risks and incorporate domain knowledge, aimed both at managers and at the 'street-level bureaucrats' on the frontlines of public service. We conclude by outlining ethical challenges and future directions for collaboration in these high-stakes applications.