Recommending Deployment Strategies for Collaborative Tasks
This work addresses the problem for requesters in crowdsourcing by providing a systematic way to deploy collaborative tasks, though it is incremental as it builds on existing optimization and computational geometry techniques.
The authors tackled the problem of recommending deployment strategies for collaborative crowdsourcing tasks by proposing StratRec, an optimization-driven middle layer that suggests strategies and parameters based on worker availability, with experiments on Amazon Mechanical Turk and synthetic data validating its qualitative and scalability aspects.
Our work contributes to aiding requesters in deploying collaborative tasks in crowdsourcing. We initiate the study of recommending deployment strategies for collaborative tasks to requesters that are consistent with deployment parameters they desire: a lower-bound on the quality of the crowd contribution, an upper-bound on the latency of task completion, and an upper-bound on the cost incurred by paying workers. A deployment strategy is a choice of value for three dimensions: Structure (whether to solicit the workforce sequentially or simultaneously), Organization (to organize it collaboratively or independently), and Style (to rely solely on the crowd or to combine it with machine algorithms). We propose StratRec, an optimization-driven middle layer that recommends deployment strategies and alternative deployment parameters to requesters by accounting for worker availability. Our solutions are grounded in discrete optimization and computational geometry techniques that produce results with theoretical guarantees. We present extensive experiments on Amazon Mechanical Turk and conduct synthetic experiments to validate the qualitative and scalability aspects of StratRec.