An Exploration of Higher Education Course Evaluation by Large Language Models
This work addresses inefficiencies in course evaluation for higher education institutions, but it is incremental as it applies existing LLM methods to a new domain without major methodological breakthroughs.
This study tackled the problem of subjective and inefficient traditional course evaluation methods in higher education by exploring the use of large language models (LLMs) for automated evaluation, finding that fine-tuned LLMs can effectively generate rational and interpretable results across 100 courses at a Chinese university.
Course evaluation is a critical component in higher education pedagogy. It not only serves to identify limitations in existing course designs and provide a basis for curricular innovation, but also to offer quantitative insights for university administrative decision-making. Traditional evaluation methods, primarily comprising student surveys, instructor self-assessments, and expert reviews, often encounter challenges, including inherent subjectivity, feedback delays, inefficiencies, and limitations in addressing innovative teaching approaches. Recent advancements in large language models (LLMs) within artificial intelligence (AI) present promising new avenues for enhancing course evaluation processes. This study explores the application of LLMs in automated course evaluation from multiple perspectives and conducts rigorous experiments across 100 courses at a major university in China. The findings indicate that: (1) LLMs can be an effective tool for course evaluation; (2) their effectiveness is contingent upon appropriate fine-tuning and prompt engineering; and (3) LLM-generated evaluation results demonstrate a notable level of rationality and interpretability.