Difficulty-level Modeling of Ontology-based Factual Questions
This work addresses the need for more sophisticated difficulty modeling in educational and professional applications, though it appears incremental as it builds on existing IRT principles with ontology-specific adaptations.
The authors tackled the problem of automatically determining the difficulty level of ontology-generated factual questions by proposing a methodology based on Item Response Theory (IRT), which considers learner proficiency and formulates ontology-based metrics to train logistic regression models for predicting difficulty across expert, intermediate, and beginner categories.
Semantics based knowledge representations such as ontologies are found to be very useful in automatically generating meaningful factual questions. Determining the difficulty level of these system generated questions is helpful to effectively utilize them in various educational and professional applications. The existing approaches for finding the difficulty level of factual questions are very simple and are limited to a few basic principles. We propose a new methodology for this problem by considering an educational theory called Item Response Theory (IRT). In the IRT, knowledge proficiency of end users (learners) are considered for assigning difficulty levels, because of the assumptions that a given question is perceived differently by learners of various proficiencies. We have done a detailed study on the features (factors) of a question statement which could possibly determine its difficulty level for three learner categories (experts, intermediates and beginners). We formulate ontology based metrics for the same. We then train three logistic regression models to predict the difficulty level corresponding to the three learner categories.