Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing
This addresses label aggregation in crowdsourcing, which is crucial for improving data quality in applications like machine learning, but it appears incremental as it builds on existing graph neural network methods.
The paper tackles the problem of aggregating noisy labels from non-expert workers in crowdsourcing by proposing a novel graph neural network framework that incorporates latent worker/task correlations, achieving superior performance over state-of-the-art models on 13 real-world datasets.
Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, an aggregation model that learns the true label by incorporating the source credibility is required. In this paper, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interaction between the same type of nodes (workers or tasks) by adding a homogeneous attention layer in the graph neural networks. Experimental results on 13 real-world datasets show superior performance over state-of-the-art models.