Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision
This addresses the challenge of enabling crowds to accomplish tightly interdependent tasks, such as creative writing, which could benefit platforms and workers seeking to scale complex work, though it is incremental in applying known principles to a new domain.
The paper tackled the problem of crowdsourcing complex creative work like fiction writing, which is interdependent, by proposing a technique that loops between reflection and revision to decompose high-level goals into actionable tasks; in a field experiment using Mechanical Novel on Amazon Mechanical Turk, it resulted in higher-quality stories than an iterative workflow.
Crowdsourcing systems accomplish large tasks with scale and speed by breaking work down into independent parts. However, many types of complex creative work, such as fiction writing, have remained out of reach for crowds because work is tightly interdependent: changing one part of a story may trigger changes to the overall plot and vice versa. Taking inspiration from how expert authors write, we propose a technique for achieving interdependent complex goals with crowds. With this technique, the crowd loops between reflection, to select a high-level goal, and revision, to decompose that goal into low-level, actionable tasks. We embody this approach in Mechanical Novel, a system that crowdsources short fiction stories on Amazon Mechanical Turk. In a field experiment, Mechanical Novel resulted in higher-quality stories than an iterative crowdsourcing workflow. Our findings suggest that orienting crowd work around high-level goals may enable workers to coordinate their effort to accomplish complex work.