EduBot -- Can LLMs Solve Personalized Learning and Programming Assignments?
This addresses the problem of personalized learning and programming assignments for students or educators, but it is an incremental approach building on existing LLM capabilities.
The paper tackles the challenge of using LLMs to solve comprehensive programming tasks with recursive requests and bug fixes, proposing EduBot, an automated assistant system that combines conceptual teaching, code development, and debugging. The result shows EduBot can complete most of 20 benchmark scenarios in less than 20 minutes, demonstrating compatibility across different LLMs.
The prevalence of Large Language Models (LLMs) is revolutionizing the process of writing code. General and code LLMs have shown impressive performance in generating standalone functions and code-completion tasks with one-shot queries. However, the ability to solve comprehensive programming tasks with recursive requests and bug fixes remains questionable. In this paper, we propose EduBot, an intelligent automated assistant system that combines conceptual knowledge teaching, end-to-end code development, personalized programming through recursive prompt-driven methods, and debugging with limited human interventions powered by LLMs. We show that EduBot can solve complicated programming tasks consisting of sub-tasks with increasing difficulties ranging from conceptual to coding questions by recursive automatic prompt-driven systems without finetuning on LLMs themselves. To further evaluate EduBot's performance, we design and conduct a benchmark suite consisting of 20 scenarios in algorithms, machine learning, and real-world problems. The result shows that EduBot can complete most scenarios in less than 20 minutes. Based on the benchmark suites, we perform a comparative study to take different LLMs as the backbone and to verify EduBot's compatibility and robustness across LLMs with varying capabilities. We believe that EduBot is an exploratory approach to explore the potential of pre-trained LLMs in multi-step reasoning and code generation for solving personalized assignments with knowledge learning and code generation.