Characterizing and modeling harms from interactions with design patterns in AI interfaces
This work addresses the oversight of interface impacts on social and ethical risks in AI systems, offering a tool for practitioners and researchers to evaluate harmful designs, though it is incremental in building on existing HCI and control theory principles.
The paper tackles the problem of harmful design patterns in AI interfaces, such as anthropomorphic or deceptive features, by proposing a conceptual model called DECAI to assess their cascading impacts, and demonstrates its application through case studies on recommendation and conversational systems.
The proliferation of applications using artificial intelligence (AI) systems has led to a growing number of users interacting with these systems through sophisticated interfaces. Human-computer interaction research has long shown that interfaces shape both user behavior and user perception of technical capabilities and risks. Yet, practitioners and researchers evaluating the social and ethical risks of AI systems tend to overlook the impact of anthropomorphic, deceptive, and immersive interfaces on human-AI interactions. Here, we argue that design features of interfaces with adaptive AI systems can have cascading impacts, driven by feedback loops, which extend beyond those previously considered. We first conduct a scoping review of AI interface designs and their negative impact to extract salient themes of potentially harmful design patterns in AI interfaces. Then, we propose Design-Enhanced Control of AI systems (DECAI), a conceptual model to structure and facilitate impact assessments of AI interface designs. DECAI draws on principles from control systems theory -- a theory for the analysis and design of dynamic physical systems -- to dissect the role of the interface in human-AI systems. Through two case studies on recommendation systems and conversational language model systems, we show how DECAI can be used to evaluate AI interface designs.