Multidimensional Item Response Theory in the Style of Collaborative Filtering
This work addresses the challenge of analyzing educational assessment data for researchers and practitioners, but it is incremental as it adapts existing collaborative filtering techniques to MIRT.
The paper tackles the problem of modeling student performance from assessment data by proposing a machine learning approach to multidimensional item response theory (MIRT), inspired by collaborative filtering, and demonstrates its application on simulated and real data, including a MOOC dataset, with efficient handling of large-scale sparse data.
This paper presents a machine learning approach to multidimensional item response theory (MIRT), a class of latent factor models that can be used to model and predict student performance from observed assessment data. Inspired by collaborative filtering, we define a general class of models that includes many MIRT models. We discuss the use of penalized joint maximum likelihood (JML) to estimate individual models and cross-validation to select the best performing model. This model evaluation process can be optimized using batching techniques, such that even sparse large-scale data can be analyzed efficiently. We illustrate our approach with simulated and real data, including an example from a massive open online course (MOOC). The high-dimensional model fit to this large and sparse dataset does not lend itself well to traditional methods of factor interpretation. By analogy to recommender-system applications, we propose an alternative "validation" of the factor model, using auxiliary information about the popularity of items consulted during an open-book exam in the course.