VCEval: Rethinking What is a Good Educational Video and How to Automatically Evaluate It
This work addresses the challenge of varying content quality in online educational videos, which is a problem for learners and educators, though it appears incremental as it builds on existing methods like language models for evaluation tasks.
The paper tackles the problem of automatically evaluating the quality of educational video content by proposing VCEval, a framework based on three principles that models the task as multiple-choice question-answering using a language model, effectively distinguishing videos of varying quality and producing interpretable results.
Online courses have significantly lowered the barrier to accessing education, yet the varying content quality of these videos poses challenges. In this work, we focus on the task of automatically evaluating the quality of video course content. We have constructed a dataset with a substantial collection of video courses and teaching materials. We propose three evaluation principles and design a new evaluation framework, \textit{VCEval}, based on these principles. The task is modeled as a multiple-choice question-answering task, with a language model serving as the evaluator. Our method effectively distinguishes video courses of different content quality and produces a range of interpretable results.