Modeling of Item-Difficulty for Ontology-based MCQs
This work addresses the challenge of automating difficulty assessment for MCQs in e-learning, reducing human effort in test authoring, but it is incremental as it builds on existing ontology-based MCQ generation methods.
The paper tackled the problem of predicting difficulty levels for ontology-based multiple choice questions (MCQs) by proposing a model that combines stem and choice set scores, incorporating test-taker skill levels. The results showed high correlation between predicted and actual difficulty levels, as validated with real students and experts using psychometric models.
Multiple choice questions (MCQs) that can be generated from a domain ontology can significantly reduce human effort & time required for authoring & administering assessments in an e-Learning environment. Even though here are various methods for generating MCQs from ontologies, methods for determining the difficulty-levels of such MCQs are less explored. In this paper, we study various aspects and factors that are involved in determining the difficulty-score of an MCQ, and propose an ontology-based model for the prediction. This model characterizes the difficulty values associated with the stem and choice set of the MCQs, and describes a measure which combines both the scores. Further more, the notion of assigning difficultly-scores based on the skill level of the test taker is utilized for predicating difficulty-score of a stem. We studied the effectiveness of the predicted difficulty-scores with the help of a psychometric model from the Item Response Theory, by involving real-students and domain experts. Our results show that, the predicated difficulty-levels of the MCQs are having high correlation with their actual difficulty-levels.