Coding for Crowdsourced Classification with XOR Queries
This work addresses the challenge of reducing query complexity in crowdsourced labeling for applications like data annotation, though it appears incremental as it builds on existing coding theory methods.
The paper tackles the problem of efficient crowdsourced classification by modeling it as a sparsely encoded source coding problem, using XOR queries to reduce the number of queries needed to almost optimal levels, with each query involving only a constant number of labels, and extends this to handle unresponsive workers and correlated labeling systems.
This paper models the crowdsourced labeling/classification problem as a sparsely encoded source coding problem, where each query answer, regarded as a code bit, is the XOR of a small number of labels, as source information bits. In this paper we leverage the connections between this problem and well-studied codes with sparse representations for the channel coding problem to provide querying schemes with almost optimal number of queries, each of which involving only a constant number of labels. We also extend this scenario to the case where some workers can be unresponsive. For this case, we propose querying schemes where each query involves only log n items, where n is the total number of items to be labeled. Furthermore, we consider classification of two correlated labeling systems and provide two-stage querying schemes with almost optimal number of queries each involving a constant number of labels.