AI incidents and 'networked trouble': The case for a research agenda
This addresses the problem of how publics can influence AI design through online controversies, proposing a new research direction rather than an incremental technical improvement.
The paper argues for a research agenda focused on AI incidents, such as a 2020 Twitter case where a user demonstrated racist bias in an image-cropping algorithm, leading Twitter to abandon that algorithm and disavow using any algorithm for the task, highlighting these incidents as a means for public participation in AI systems.
Against a backdrop of widespread interest in how publics can participate in the design of AI, I argue for a research agenda focused on AI incidents - examples of AI going wrong and sparking controversy - and how they are constructed in online environments. I take up the example of an AI incident from September 2020, when a Twitter user created a 'horrible experiment' to demonstrate the racist bias of Twitter's algorithm for cropping images. This resulted in Twitter not only abandoning its use of that algorithm, but also disavowing its decision to use any algorithm for the task. I argue that AI incidents like this are a significant means for participating in AI systems that require further research. That research agenda, I argue, should focus on how incidents are constructed through networked online behaviours that I refer to as 'networked trouble', where formats for participation enable individuals and algorithms to interact in ways that others - including technology companies - come to know and come to care about. At stake, I argue, is an important mechanism for participating in the design and deployment of AI.