Cognitive Diagnosis with Explicit Student Vector Estimation and Unsupervised Question Matrix Learning
This work addresses limitations in educational cognitive diagnosis models by reducing reliance on expert-labeled data and strong probabilistic assumptions, though it appears incremental as it builds upon the classical DINA model.
The paper tackles the problem of cognitive diagnosis in education by proposing an explicit student vector estimation method to relax strong assumptions in the DINA model and an unsupervised algorithm to automatically label the Q-matrix, showing improved accuracy over DINA and comparable performance to manually labeled Q-matrices on real-world datasets.
Cognitive diagnosis is an essential task in many educational applications. Many solutions have been designed in the literature. The deterministic input, noisy "and" gate (DINA) model is a classical cognitive diagnosis model and can provide interpretable cognitive parameters, e.g., student vectors. However, the assumption of the probabilistic part of DINA is too strong, because it assumes that the slip and guess rates of questions are student-independent. Besides, the question matrix (i.e., Q-matrix) recording the skill distribution of the questions in the cognitive diagnosis domain often requires precise labels given by domain experts. Thus, we propose an explicit student vector estimation (ESVE) method to estimate the student vectors of DINA with a local self-consistent test, which does not rely on any assumptions for the probabilistic part of DINA. Then, based on the estimated student vectors, the probabilistic part of DINA can be modified to a student dependent model that the slip and guess rates are related to student vectors. Furthermore, we propose an unsupervised method called heuristic bidirectional calibration algorithm (HBCA) to label the Q-matrix automatically, which connects the question difficulty relation and the answer results for initialization and uses the fault tolerance of ESVE-DINA for calibration. The experimental results on two real-world datasets show that ESVE-DINA outperforms the DINA model on accuracy and that the Q-matrix labeled automatically by HBCA can achieve performance comparable to that obtained with the manually labeled Q-matrix when using the same model structure.