Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis
This work addresses the need for personalized support in electrical engineering education, though it is incremental as it builds on existing GPT-4o capabilities.
The researchers tackled the problem of improving large language models for automated homework assessment in undergraduate circuit analysis, resulting in an increase in correct response rate from 74.71% to 97.70%.
This research full paper presents an enhancement pipeline for large language models (LLMs) in assessing homework for an undergraduate circuit analysis course, aiming to improve LLMs' capacity to provide personalized support to electrical engineering students. Existing evaluations have demonstrated that GPT-4o possesses promising capabilities in assessing student homework in this domain. Building on these findings, we enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints. These strategies effectively address common errors observed in GPT-4o's responses when using simple prompts, leading to a substantial improvement in assessment accuracy. Specifically, the correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data on entry-level circuit analysis topics. This work lays a foundation for the effective integration of LLMs into circuit analysis instruction and, more broadly, into engineering education.