Ontologies in Design: How Imagining a Tree Reveals Possibilites and Assumptions in Large Language Models
This work addresses the need for better understanding and designing sociotechnical systems like LLMs by focusing on ontological assumptions, offering a novel framework for researchers and practitioners, though it is incremental in building on existing critical analyses.
The paper tackles the problem of analyzing large language models (LLMs) by emphasizing ontologies as a key dimension beyond values like bias, proposing four orientations (pluralism, groundedness, liveliness, enactment) for design, and demonstrates this through analyses of chatbot responses and agent simulations.
Amid the recent uptake of Generative AI, sociotechnical scholars and critics have traced a multitude of resulting harms, with analyses largely focused on values and axiology (e.g., bias). While value-based analyses are crucial, we argue that ontologies -- concerning what we allow ourselves to think or talk about -- is a vital but under-recognized dimension in analyzing these systems. Proposing a need for a practice-based engagement with ontologies, we offer four orientations for considering ontologies in design: pluralism, groundedness, liveliness, and enactment. We share examples of potentialities that are opened up through these orientations across the entire LLM development pipeline by conducting two ontological analyses: examining the responses of four LLM-based chatbots in a prompting exercise, and analyzing the architecture of an LLM-based agent simulation. We conclude by sharing opportunities and limitations of working with ontologies in the design and development of sociotechnical systems.