Cracking CodeWhisperer: Analyzing Developers' Interactions and Patterns During Programming Tasks
This research addresses the adoption of AI tools by software developers, providing insights into user behavior, but it is incremental as it focuses on analyzing existing interactions without proposing new methods.
The study investigated how software developers interact with Amazon's CodeWhisperer, an AI code-generation tool, identifying four behavioral patterns such as incremental code refinement and explicit instruction using natural language comments.
The use of AI code-generation tools is becoming increasingly common, making it important to understand how software developers are adopting these tools. In this study, we investigate how developers engage with Amazon's CodeWhisperer, an LLM-based code-generation tool. We conducted two user studies with two groups of 10 participants each, interacting with CodeWhisperer - the first to understand which interactions were critical to capture and the second to collect low-level interaction data using a custom telemetry plugin. Our mixed-methods analysis identified four behavioral patterns: 1) incremental code refinement, 2) explicit instruction using natural language comments, 3) baseline structuring with model suggestions, and 4) integrative use with external sources. We provide a comprehensive analysis of these patterns .