MiSC: Mixed Strategies Crowdsourcing
This work addresses label aggregation in crowdsourcing, offering a versatile framework that combines existing approaches, though it appears incremental as it integrates rather than introduces a new paradigm.
The paper tackles the problem of integrating conventional crowdsourcing and tensor completion techniques for label aggregation, proposing the MiSC framework which outperforms state-of-the-art methods in experiments.
Popular crowdsourcing techniques mostly focus on evaluating workers' labeling quality before adjusting their weights during label aggregation. Recently, another cohort of models regard crowdsourced annotations as incomplete tensors and recover unfilled labels by tensor completion. However, mixed strategies of the two methodologies have never been comprehensively investigated, leaving them as rather independent approaches. In this work, we propose $\textit{MiSC}$ ($\textbf{Mi}$xed $\textbf{S}$trategies $\textbf{C}$rowdsourcing), a versatile framework integrating arbitrary conventional crowdsourcing and tensor completion techniques. In particular, we propose a novel iterative Tucker label aggregation algorithm that outperforms state-of-the-art methods in extensive experiments.