Learning from Teaching Assistants to Program with Subgoals: Exploring the Potential for AI Teaching Assistants
This addresses the challenge of scalable personalized support for novice programming learners, though it's an incremental application of existing AI models.
The study investigated using generative AI as teaching assistants in introductory programming education, finding that learners solved tasks faster with comparable scores to human TAs and had similar perceptions of helpfulness and satisfaction.
With recent advances in generative AI, conversational models like ChatGPT have become feasible candidates for TAs. We investigate the practicality of using generative AI as TAs in introductory programming education by examining novice learners' interaction with TAs in a subgoal learning environment. To compare the learners' interaction and perception of the AI and human TAs, we conducted a between-subject study with 20 novice programming learners. Learners solve programming tasks by producing subgoals and subsolutions with the guidance of a TA. Our study shows that learners can solve tasks faster with comparable scores with AI TAs. Learners' perception of the AI TA is on par with that of human TAs in terms of speed and comprehensiveness of the replies and helpfulness, difficulty, and satisfaction of the conversation. Finally, we suggest guidelines to better design and utilize generative AI as TAs in programming education from the result of our chat log analysis.