Annotating Implicit Reasoning in Arguments with Causal Links
This work addresses the challenge of understanding implicit reasoning in arguments for natural language processing and argumentation analysis, but it is incremental as it builds on existing schemes like Argument from Consequences.
The paper tackled the problem of identifying implicit reasoning links between argument components by proposing a semi-structured template based on causality, and found substantial inter-annotator agreement but raised questions about the feasibility of high-quality crowdsourcing.
Most of the existing work that focus on the identification of implicit knowledge in arguments generally represent implicit knowledge in the form of commonsense or factual knowledge. However, such knowledge is not sufficient to understand the implicit reasoning link between individual argumentative components (i.e., claim and premise). In this work, we focus on identifying the implicit knowledge in the form of argumentation knowledge which can help in understanding the reasoning link in arguments. Being inspired by the Argument from Consequences scheme, we propose a semi-structured template to represent such argumentation knowledge that explicates the implicit reasoning in arguments via causality. We create a novel two-phase annotation process with simplified guidelines and show how to collect and filter high-quality implicit reasonings via crowdsourcing. We find substantial inter-annotator agreement for quality evaluation between experts, but find evidence that casts a few questions on the feasibility of collecting high-quality semi-structured implicit reasoning through our crowdsourcing process. We release our materials(i.e., crowdsourcing guidelines and collected implicit reasonings) to facilitate further research towards the structured representation of argumentation knowledge.