No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot
For developers and tool designers, this work highlights that cognitive diversity leads to varied needs in conversational programming assistants, which current tools may not fully address.
This study investigates how cognitive diversity (problem-solving styles and experience) affects developers' interactions with GitHub Copilot Chat, identifying 5 interaction modes and 10 underlying needs through a think-aloud study with 27 participants. Results show that problem-solving styles and experience shape these modes and needs, providing recommendations for designing more inclusive programming assistants.
Conversational LLM-based ``programming assistants'' provide a range of benefits to developers. However, recent studies demonstrate the variety in individual developers' needs regarding programming assistants, and challenges encountered by only specific groups of developers. In this study, we explore the role of cognitive diversity in shaping interactions with GitHub Copilot chat. Through a mixed-methods think aloud study with 27 professional developers and students, we characterize 5 distinct ``interaction modes'' and 10 underlying needs in developers' interactions, forming a conceptual model. We characterize links between these modes, needs, and developers' problem-solving styles and experience profiles, showing how cognitive diversity may shape developers' interactions. We provide insights and recommendations for researchers and practitioners on how to design, research, and employ programming assistants to better account for diverse developer needs.