Enhancing Item Response Theory for Cognitive Diagnosis
This work addresses the problem of inaccurate and limited cognitive diagnosis in educational applications, such as adaptive testing, by enhancing IRT with deep learning, though it is incremental as it builds on existing IRT methods.
The authors tackled the limitations of traditional Item Response Theory (IRT) in cognitive diagnosis by proposing a Deep Item Response Theory (DIRT) framework that leverages question text semantics with deep learning, resulting in improved effectiveness and interpretability as demonstrated on real-world data.
Cognitive diagnosis is a fundamental and crucial task in many educational applications, e.g., computer adaptive test and cognitive assignments. Item Response Theory (IRT) is a classical cognitive diagnosis method which can provide interpretable parameters (i.e., student latent trait, question discrimination, and difficulty) for analyzing student performance. However, traditional IRT ignores the rich information in question texts, cannot diagnose knowledge concept proficiency, and it is inaccurate to diagnose the parameters for the questions which only appear several times. To this end, in this paper, we propose a general Deep Item Response Theory (DIRT) framework to enhance traditional IRT for cognitive diagnosis by exploiting semantic representation from question texts with deep learning. In DIRT, we first use a proficiency vector to represent students' proficiency in knowledge concepts and embed question texts and knowledge concepts to dense vectors by Word2Vec. Then, we design a deep diagnosis module to diagnose parameters in traditional IRT by deep learning techniques. Finally, with the diagnosed parameters, we input them into the logistic-like formula of IRT to predict student performance. Extensive experimental results on real-world data clearly demonstrate the effectiveness and interpretation power of DIRT framework.