MmmTurkey: A Crowdsourcing Framework for Deploying Tasks and Recording Worker Behavior on Amazon Mechanical Turk
This work addresses a specific problem for researchers using crowdsourcing platforms, but it is incremental as it builds on existing external HIT methods.
The researchers tackled the problem of creating and managing external HITs on Amazon Mechanical Turk, which is often redundant and time-consuming, by developing MmmTurkey, a framework that enables quick creation and management of these tasks while capturing detailed worker behavioral data.
Internal HITs on Mechanical Turk can be programmatically restrictive, and as a result, many requesters turn to using external HITs as a more flexible alternative. However, creating such HITs can be redundant and time-consuming. We present MmmTurkey, a framework that enables researchers to not only quickly create and manage external HITs, but more significantly also capture and record detailed worker behavioral data characterizing how each worker completes a given task.