HCAIIRLGMar 15

To LLM, or Not to LLM: How Designers and Developers Navigate LLMs as Tools or Teammates

MITOxford
arXiv:2604.1534428.22 citationsh-index: 6
AI Analysis

This addresses the sociotechnical positioning problem for designers and developers in integrating LLMs into organizational workflows, though it is incremental as it builds on existing human-AI interaction research.

The study investigated how designers and developers decide whether to use LLMs as tools or teammates in workflows, finding that framing LLMs as tools under human control was acceptable, while teammate framings with ambiguous agency caused hesitation, but productive configurations existed with explicit oversight.

Large language models (LLMs) are increasingly integrated into design and development workflows, yet decisions about their use are rarely binary or purely technical. We report findings from a constructivist grounded theory study based on interviews with 33 designers and developers across three large technology organisations. Rather than evaluating LLMs solely by capability, participants reasoned about the role an LLM could occupy within a workflow and how that role would interact with existing structures of responsibility and organisational accountability. When LLMs were framed as tools under clear human control, their use was typically acceptable and could be integrated within existing governance structures. When framed as teammates with shared or ambiguous agency, practitioners expressed hesitation, particularly when responsibility for outcomes could not be clearly justified. At the same time, participants also described productive teammate configurations in which LLMs supported collaborative reasoning while remaining embedded within explicit oversight structures. We identify tool and teammate framings as recurring ways in which designers and developers position LLMs relative to human work and present an analytic rubric describing how role framing shapes decision authority, accountability ownership, oversight strategies, and organisational acceptability. By foregrounding design-time reasoning, this work reframes To LLM or Not to LLM as a sociotechnical positioning problem that emerges during system design rather than during post-deployment evaluation.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes