Supervised Collective Classification for Crowdsourcing
This work addresses the challenge of data quality in crowdsourcing for collective classification, though it appears incremental as it builds on existing methods.
The paper tackles the problem of improving classification accuracy in crowdsourcing by proposing a supervised collective classification algorithm that identifies reliable labelers from training data, achieving better accuracy than unsupervised methods.
Crowdsourcing utilizes the wisdom of crowds for collective classification via information (e.g., labels of an item) provided by labelers. Current crowdsourcing algorithms are mainly unsupervised methods that are unaware of the quality of crowdsourced data. In this paper, we propose a supervised collective classification algorithm that aims to identify reliable labelers from the training data (e.g., items with known labels). The reliability (i.e., weighting factor) of each labeler is determined via a saddle point algorithm. The results on several crowdsourced data show that supervised methods can achieve better classification accuracy than unsupervised methods, and our proposed method outperforms other algorithms.