How To Grade a Test Without Knowing the Answers --- A Bayesian Graphical Model for Adaptive Crowdsourcing and Aptitude Testing
This work addresses the challenge of resource allocation in aptitude testing and crowdsourcing, offering a more efficient approach for educators and platform managers, though it is incremental in its adaptation of existing probabilistic models.
The authors tackled the problem of efficiently grading tests and crowdsourcing tasks without known answers by proposing a Bayesian graphical model that jointly estimates question difficulties, participant abilities, and correct answers, and they demonstrated that their adaptive testing scheme reduces the number of questions needed to achieve the same accuracy as static methods.
We propose a new probabilistic graphical model that jointly models the difficulties of questions, the abilities of participants and the correct answers to questions in aptitude testing and crowdsourcing settings. We devise an active learning/adaptive testing scheme based on a greedy minimization of expected model entropy, which allows a more efficient resource allocation by dynamically choosing the next question to be asked based on the previous responses. We present experimental results that confirm the ability of our model to infer the required parameters and demonstrate that the adaptive testing scheme requires fewer questions to obtain the same accuracy as a static test scenario.