Efficient Crowd Exploration of Large Networks: The Case of Causal Attribution
This work addresses the problem of efficiently crowdsourcing complex networks for researchers and practitioners in fields like causal inference, though it is incremental as it builds on existing crowdsourcing methods.
The paper tackles the challenge of crowdsourcing large networks through microtasks, focusing on causal attribution networks, and introduces Iterative Pathway Refinement to efficiently explore these networks, with experiments on Amazon Mechanical Turk showing successful extraction and analysis of a large-scale network.
Accurately and efficiently crowdsourcing complex, open-ended tasks can be difficult, as crowd participants tend to favor short, repetitive "microtasks". We study the crowdsourcing of large networks where the crowd provides the network topology via microtasks. Crowds can explore many types of social and information networks, but we focus on the network of causal attributions, an important network that signifies cause-and-effect relationships. We conduct experiments on Amazon Mechanical Turk (AMT) testing how workers propose and validate individual causal relationships and introduce a method for independent crowd workers to explore large networks. The core of the method, Iterative Pathway Refinement, is a theoretically-principled mechanism for efficient exploration via microtasks. We evaluate the method using synthetic networks and apply it on AMT to extract a large-scale causal attribution network, then investigate the structure of this network as well as the activity patterns and efficiency of the workers who constructed this network. Worker interactions reveal important characteristics of causal perception and the network data they generate can improve our understanding of causality and causal inference.