Une mesure d'expertise pour le crowdsourcing
This addresses quality control in crowdsourcing campaigns, but it is incremental as it focuses on a specific scenario with known gold data.
The paper tackles the problem of evaluating participant work quality in crowdsourcing, particularly for small, fast, non-automatable tasks, by proposing a method to calculate expertise degrees using belief function theory and graph-based data structuring, applied to data with known gold truth.
Crowdsourcing, a major economic issue, is the fact that the firm outsources internal task to the crowd. It is a form of digital subcontracting for the general public. The evaluation of the participants work quality is a major issue in crowdsourcing. Indeed, contributions must be controlled to ensure the effectiveness and relevance of the campaign. We are particularly interested in small, fast and not automatable tasks. Several methods have been proposed to solve this problem, but they are applicable when the "golden truth" is not always known. This work has the particularity to propose a method for calculating the degree of expertise in the presence of gold data in crowdsourcing. This method is based on the belief function theory and proposes a structuring of data using graphs. The proposed approach will be assessed and applied to the data.