Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programming
This work addresses the problem of understanding collaboration dynamics in programming for developers and researchers, but it is incremental as it synthesizes existing literature without new empirical results.
The paper compares human-human and human-AI pair programming, finding mixed effectiveness in both approaches, with measures for human-AI being less comprehensive, and identifies moderating factors like mismatched expertise that could guide future AI assistant design.
The emergence of large-language models (LLMs) that excel at code generation and commercial products such as GitHub's Copilot has sparked interest in human-AI pair programming (referred to as "pAIr programming") where an AI system collaborates with a human programmer. While traditional pair programming between humans has been extensively studied, it remains uncertain whether its findings can be applied to human-AI pair programming. We compare human-human and human-AI pair programming, exploring their similarities and differences in interaction, measures, benefits, and challenges. We find that the effectiveness of both approaches is mixed in the literature (though the measures used for pAIr programming are not as comprehensive). We summarize moderating factors on the success of human-human pair programming, which provides opportunities for pAIr programming research. For example, mismatched expertise makes pair programming less productive, therefore well-designed AI programming assistants may adapt to differences in expertise levels.