Misspecifying non-compensatory as compensatory IRT: analysis of estimated skills and variance
This work addresses a methodological issue in educational testing and psychometrics, providing insights for researchers and practitioners using IRT models, though it is incremental as it builds on prior findings.
The study investigated the consequences of misspecifying non-compensatory as compensatory models in multidimensional item response theory, finding that it leads to underestimation of higher skills and newly discovered overestimation around the origin, with analysis showing differences in asymptotic variance of estimated parameters.
Multidimensional item response theory is a statistical test theory used to estimate the latent skills of learners and the difficulty levels of problems based on test results. Both compensatory and non-compensatory models have been proposed in the literature. Previous studies have revealed the substantial underestimation of higher skills when the non-compensatory model is misspecified as the compensatory model. However, the underlying mechanism behind this phenomenon has not been fully elucidated. It remains unclear whether overestimation also occurs and whether issues arise regarding the variance of the estimated parameters. In this paper, we aim to provide a comprehensive understanding of both underestimation and overestimation through a theoretical approach. In addition to the previously identified underestimation of the skills, we newly discover that the overestimation of skills occurs around the origin. Furthermore, we investigate the extent to which the asymptotic variance of the estimated parameters differs when considering model misspecification compared to when it is not taken into account.