AI incidents and ‘networked trouble’: The case for a research agenda

Abstract

Against a backdrop of widespread interest in how publics can participate in the design of AI, I argue for a research agenda focused on AI incidents – examples of AI going wrong and sparking controversy – and how they are constructed in online environments. I take up the example of an AI incident from September 2020, when a Twitter user created a ‘horrible experiment’ to demonstrate the racist bias of Twitter's algorithm for cropping images. This resulted in Twitter not only abandoning its use of that algorithm, but also disavowing its decision to use any algorithm for the task. I argue that AI incidents like this are a significant means for participating in AI systems that require further research. That research agenda, I argue, should focus on how incidents are constructed through networked online behaviours that I refer to as ‘networked trouble’, where formats for participation enable individuals and algorithms to interact in ways that others – including technology companies – come to know and come to care about. At stake, I argue, is an important mechanism for participating in the design and deployment of AI.