Bayes Nets in Educational Assessment: Where Do the Numbers Come From?
This work addresses the problem of designing advanced educational assessments for educators and researchers, but it is incremental as it builds on existing Bayesian and MCMC methods.
The paper tackles the challenge of complex educational assessments by proposing Bayesian inference networks (BINs) within the Portal framework to update beliefs about students' knowledge based on observations, using MCMC techniques to estimate probabilities from data and expert judgment, with a numerical example provided for latent class modeling.
As observations and student models become complex, educational assessments that exploit advances in technology and cognitive psychology can outstrip familiar testing models and analytic methods. Within the Portal conceptual framework for assessment design, Bayesian inference networks (BINs) record beliefs about students' knowledge and skills, in light of what they say and do. Joining evidence model BIN fragments- which contain observable variables and pointers to student model variables - to the student model allows one to update belief about knowledge and skills as observations arrive. Markov Chain Monte Carlo (MCMC) techniques can estimate the required conditional probabilities from empirical data, supplemented by expert judgment or substantive theory. Details for the special cases of item response theory (IRT) and multivariate latent class modeling are given, with a numerical example of the latter.